Pseudoscience and bad science in biomedicine

Pseudoscience (false science) and science based on faulty and biased studies (bad science) produce false or uncertain knowledge, with poor or no evidence. Both represent a health risk: pseudoscience-based therapies because they can replace or delay conventional treatments, and low-quality biomedicine because it promotes medical interventions that can be dangerous. In the press, alternative therapies are less prevalent than low-quality research, while the former tends to be framed negatively and the latter favourably. Both require more thorough and rigorous studies to better understand their negative effects on critical thinking, economics, and health-related decision making.

Keywords: pseudoscience, bad science, biomedicine, complementary and alternative medicine, scientific communication.

«False sciences have proliferated under the umbrella of science as a counterfeit does at the expense of the original product»

The undeserved predicament of pseudoscience or false science is an undesirable effect of the prestige of modern science. False sciences have proliferated under the umbrella of science as a counterfeit does at the expense of the original product. At the same time, what has been called «bad science» or science of low methodological quality has also thrived. In spite of their differences, pseudoscience, which passes for a science without being one, and bad science, which is presented as rigorous science but is not so, have something in common: the production of false, uncertain, or doubtful ideas and knowledge. In both cases, this knowledge is not ideal for making well-informed decisions, especially when these can affect health. Given that, in networks and the media, pseudoscience coexists with bad science, given that it is not easy for many people to tell their messages apart from each other, and from those of quality science, and given that this information can be harmful to people’s health, it seems justified to analyse pseudoscience and bad science together.

There are many ways of doing bad science, from falsifying data to malpractice caused by the accumulation of errors and biases, both voluntary and involuntary. But the problem with bad science, popularised by Ben Goldacre in a column for The Guardian and later in a book (Goldacre, 2009), is not yet well defined and characterised. Although to varying degrees, all forms of bad science share shortcomings and weaknesses in the application of the scientific method and are characterised by uncertain or unreliable results.

«Pseudoscience and bad science have something in common: the production of false, uncertain, or doubtful ideas and knowledge»

Pseudoscience and bad science, despite their radical differences, have some common negative effects that affect the culture, economy, and well-being of the citizenry. This article aims to review some of these effects, in particular the production of false, unproven, or low-quality knowledge; the potential health risks of pseudoscientific therapies and poor biomedical science; and their presence in the press, since the media are a means for the dissemination of both types of messages. This work focuses on biomedicine for three reasons: In the medical field, there is an abundance of pseudoscientific therapies, generally called alternative and complementary medicine. Although medicine is not strictly speaking a science, but rather an applied science or, if one wants, a technology, most citizens consider it the prototype of science (Bauer, 1998). Additionally, biomedicine, i.e., clinical disciplines plus the natural and social sciences that make up the scientific corpus of medicine, represents around half of the scientific production (Camí, Méndez-Vásquez, & Suñén-Pinyol, 2008) and a good part of the science news.

When some of the most popular therapies, such as homeopathy or acupuncture, have been subjected to controlled clinical trials for the treatment of different ailments, the promised efficacy has generally been shown to be false. / Mètode

Uncertain or false knowledge

Since the 1990s, evidence-based medicine has become the benchmark for guiding biomedical research and clinical practice. The antithesis of this approach is medicine based not on scientific evidence but on other elements such as tradition, experience, fashion, authority, marketing, and pseudoscience. In contrast to evidence-based medicine, we can distinguish, on the one hand, complementary and alternative medicine, which is based on pseudoscience; and, on the other hand, evidence-biased medicine, which would be bad science as applied to medicine.

Out of all the possible questions about health and disease, biomedical science has only answered a small number of them. Moreover, these findings still have a very variable degree of certainty and need to be discriminated against to help make decisions that affect individual and collective health. In 2004, GRADE (Grading of Recommendations, Assessment, Development, and Evaluation) was consolidated as a standard system to allow the quality of evidence for the effects of medical interventions to be ranked into four categories: high (implying that the confidence in the effects-estimates will hardly change even if more studies are done in the future), moderate (estimates may change with new studies), low (it is very likely that future estimates will be different), and very low (the estimate of an intervention’s effect is uncertain; Guyatt et al., 2008). The categorisation depends not only on whether the studies are experimental (clinical trials) or observational (clinical trials are considered to be of high quality and observational studies, of low quality), but also on various factors and biases that change the quality of the results. GRADE is an objective, exhaustive, and transparent system for ranking evidence, which makes it possible to discriminate against poor or very low-quality science.

Neither complementary and alternative medicine nor bad science applied to medicine provide sufficiently reliable data to be able to recommend the implementation of medical interventions or to make decisions that affect one’s health. But the production of such false or uncertain data is very different.

The problem with bad science, popularised by Ben Goldacre in a column for The Guardian and later in a book, is not yet well defined and characterised. / Mètode

Medicine without any scientific evidence

Pseudoscientific therapies are characterised not only because their efficacy and safety have usually not been scientifically proven (as required by the FDA and the EMEA, the US and European drug agencies, respectively), but also because their proponents do not necessarily even believe that this is necessary. On the contrary, they consider that the scientific method is not applicable to their remedies and that only the support of anecdotes and ideas that ignore biological mechanisms are required (Angell & Kassirer, 1998).

Most complementary and alternative therapies have not undergone clinical trials to determine their efficacy and safety, so their therapeutic promises must be considered unproven ideas. When some of the most popular therapies, such as homeopathy or acupuncture, have been subjected to controlled clinical trials for the treatment of different ailments, the promised efficacy has generally been shown to be false. In most clinical trials, the efficacy has been found to be similar to a placebo. In some cases, and for some minor ailments, a few trials have shown some favourable evidence for efficacy, but, in general, these are low methodological quality trials, and so we still lack rigorous confirmatory studies (Ministerio de Sanidad, Política Social e Igualdad, 2011).

«In contrast to evidence-based medicine, we can distinguish, on the one hand, complementary and alternative medicine and evidence-biased medicine»

If at some point any of the alternative therapies were to prove effective against any ailment, it would automatically become evidence-based therapy and would cease to be an alternative therapy (Angell & Kassirer, 1998). However, for the most part this has not happened, and these therapies must be considered treatments of unproven efficacy. As the editorialists in The New England Journal of Medicine Marcia Angell and Jerome Kassirer state, there is only medicine that has been adequately tested and medicine that has not, medicine that works and medicine that may or may not work.

Therapies of uncertain efficacy are not only found in the field of pseudoscience: The effectiveness of almost half of the 3,000 treatments used in conventional medicine is unknown, according to an analysis of controlled and randomised clinical trials published on the BMJ Clinical Evidence website, one of the journals associated with the British Medical Journal. However, this analysis does not indicate the extent to which treatments of unknown efficacy are used (BMJ Publishing Group Limited, 2017). But, unlike what happens in the field of pseudoscience, the scientific community recognises that this lack of evidence must be corrected.

False results and poor-quality evidence

The problem with bad biomedical science is not so much a lack of evidence as a lack of false results and that there are low levels of certainty in poor-quality research. The publication of the article Why most published research findings are false (Ioannidis, 2005) shows that the probability of obtaining false results increases, among other factors, with the scarcity of studies that address a research question, the lesser magnitude of the effect, the existence of conflicts of interest and, in general, with greater flexibility in the design, definitions, and study variables. This paper suggests that half of the results of biomedical research are false or misinterpreted, with significant biases that favour prejudice. Although Ioannidis’s study has been widely criticised and nuanced (Djulbegovic & Hozo, 2007; Goodman & Greenland, 2007), its controversial estimation has been a blow to the consciousness of the scientific community and has triggered a cascade of studies on the quality of research and meta-science.

«Therapies of uncertain efficacy are not only found in the field of pseudoscience»

One of the most widespread methodological deficiencies in biomedicine is the interpretation of research results based on statistical significance alone, typically on probability (p)-values lower than 0.05 (Ioannidis, 2005). The emphasis on the p-value has favoured techniques such as p-hacking or data fishing (selecting non-significant data until the results become significant). This has inflated the number of «irrelevant studies» (Ioannidis, 2005), because statistical significance does not necessarily imply scientific or, much less, clinical relevance. It is estimated that of the 50,000 articles published each year in the top 110 medical journals, only about 3,000 (6 %) follow correct methodology and provide information relevant to clinical practice (Haynes, 2005).

This problem has been compounded in recent years by the fact that the results of many investigations are not reproducible. A Nature survey among 1,576 researchers has shown that 52 % of those surveyed consider that science to be experiencing a reproducibility crisis, although less than 31 % consider that the impossibility of reproducing the results implies that they are false (Baker, 2016). The survey suggests that there is a gradient, ranging from the lowest irreproducibility levels of chemistry, physics, and engineering to the highest levels in biology, medicine, and other disciplines. Factors that explain this problem include shortcomings in study design, selective publication of results, errors in statistical analysis, selection of unrepresentative samples, and other methodological deficiencies, in addition to peer-review limitations.

The influence of diet on health is very difficult to control in epidemiological studies. Especially eloquent are the results of observational studies on diet and cancer, because for almost all food items it is possible to find both studies that associate them with an increased risk of cancer or with a reduction in risk. / Congerdesign

Dietetics as an example

One area that is particularly prone to bad science and pseudoscience is dietetics. The influence of diet on health is very difficult to control in epidemiological studies; in addition, many of the research projects are sponsored by the food industry, known as Big Food, so major conflicts of interest exist. Especially eloquent are the results of observational studies on diet and cancer, because for almost all food items it is possible to find both studies that associate them with an increased risk of cancer and studies that associate them with a reduction in risk (Schoenfeld & Ioannidis, 2013).

Diet is considered a central element of health, but many dietary questions lack a scientific answer. Existing evidence tends to be of low or very low quality, with most nutrition studies being observational, which are unreliable from the outset. Therefore, it is not uncommon for dietitians and nutritionists to overstep their bounds when making recommendations in the name of science. For Goldacre (2009, p. 86), they make four mistakes in interpreting the scientific literature: they extrapolate laboratory results to make claims about human diet, they extrapolate observational data to formulate dietary interventions, they selectively collect evidence (cherry picking), and they talk about scientific evidence that does not actually exist.

This lack of evidence, due in part to the intrinsic difficulties of nutrition research, to the low quality of evidence, and to conflicts of interest, facilitates the proliferation of miracle diets and dubious or pseudoscientific recommendations, which can have a negative impact on health.

Health risks

The treatment of some diseases with complementary and alternative medicine is not without risks for the patient (Angell & Kassirer, 1998). In Spain, of 139 such therapies identified by the Ministry of Health (2011), 34 may pose a health risk. This risk is particularly important in the case of serious diseases, such as cancer, where delays in treatment or rejection of conventional therapies in favour of other alternatives may even endanger the life of the patient.

A recent study with cancer patients who initially choose to be treated with alternative therapies has shown that they have a higher risk of death than those patients who resort to conventional treatments (chemotherapy, radiotherapy, surgery, and/or hormone therapy) from the beginning (Johnson, Park, Gross, & Yu, 2018). The study, conducted with patients suffering from the four most common cancers (breast, prostate, lung, and intestine), shows that the risk of death was five times higher for patients with breast cancer, four times higher for intestinal cancer, and twice as high for lung cancer, while those with prostate cancer had a similar risk.

The health risks of bad biomedical science have been less studied than those of complementary and alternative medicine. But one example of its potential risk is the use of female hormone replacement therapy in the 1990s. Although this treatment was supported by observational studies that indicated that it reduced vascular risk, we had to wait until the 21st century to confirm that this was not the case and that hormone therapy entails health risks that, in many cases, do not outweigh its benefits. Had GRADE existed at that time, it would have been possible to verify that the evidence in its favour was very low quality because it had come from observational studies with inconsistent results (Guyatt et al., 2008), and this therapy, based on bad science and which was still in clinical trials at the time, would not have been so widely recommended by doctors and scientific associations.

«One area that is particularly prone to bad science and pseudoscience is dietetics»

Media dissemination

Although there is a clear epistemological difference between pseudoscience and bad science, in practice, for citizens, the limits are blurred because pseudoscientific messages coexist on the Internet alongside those based on bad science. A large part of the public lacks the necessary criteria to identify them and so, are exposed to these two potential threats. The media, which should represent a filter for the quality of information, do not entirely fulfil this function. On the one hand, the lack of scientific culture among those responsible for the media facilitates the spread of pseudosciences (Cortiñas-Rovira, Alonso-Marcos, Pont-Sorribes, & Escribà-Sales, 2015), and on the other hand, communicators and journalists contribute to the spread of false results and bad science (Dumas-Mallet, Smith, Boraud, & Gonon, 2017; Sumner et al., 2016).

Of the five most widely used therapies in Spain, yoga is the only one with significant coverage in the digital press; acupuncture, chiropractic techniques, homeopathy, and reflexology have a very low presence. / Mètode

Pseudoscience in the press

The presence of pseudosciences in the media has been studied in a superficial and fragmented way. As far as complementary and alternative medicine is concerned, its visibility in the Spanish press seems to be insignificant. Of the five most widely used therapies in Spain (Ministerio de Sanidad, Política Social e Igualdad, 2011), yoga is the only one with significant coverage in the digital press; the presence of the next four (acupuncture, chiropractic techniques, homeopathy, and reflexology) is very low (Moreno-Castro & Lopera-Pareja, 2016).

In the most widely distributed quality press in the United Kingdom and Spain, the presence of texts that delve into the debate on the effectiveness and safety of homeopathy is scarce (barely one article a month). Most of these journalistic texts are against the supposed efficacy and innocuousness of homeopathy. However, these media are not very committed to transferring a rigorous debate on homeopathy to citizens (Escribà-Sales, Cortiñas Rovira, & Alonso-Marcos, 2015).

In the most widely distributed quality press in the United Kingdom and Spain, the presence of texts that delve into the debate on the effectiveness and safety of homeopathy is scarce. / Mètode

An analysis of the coverage of complementary and alternative medicine in the Australian press shows the discreet presence of these therapies and an inconsistent approach. Positive frameworks are twice as frequent as negative ones; the most common are those that present them as complementary medicine but not as alternatives to conventional treatments, and those that present it as regular and necessary medicine. Less common are approaches that emphasise their potential health risks or present them as self-serving, dangerous quackery (Mercury & Eliott, 2011).

Bad science in the press

Although there are no specific studies on bad science in the press, there are indirect data showing that studies of higher quality are not exactly the most relevant in general information newspapers. Among the biomedical news headlines featured on newspaper covers reporting on lower quality research: 46 % refer to observational studies, compared to 21 % reporting on clinical trials, and 3 % in reference to systematic reviews of clinical trials; the rest are news that deal with animal or laboratory studies (17 %) and expert opinions (14 %) (Lai & Lane, 2009).

«The media are not very committed to transferring a rigorous debate on homeopathy to citizens»

Whether Ioannidis’s (2005) estimate that half of the research results are false is closer to or further from the truth, it seems clear that the proportion of false news reporting about research is even higher. While science is wary of newer studies, precisely because they are more likely to be false, journalism seeks them out and pays more attention to them. This media preference for novel, less contrasted results promotes the concentration and spread of bad science (Dumas-Mallet et al., 2017). In addition, both the communicators who produce press releases and the journalists themselves can add an extra element of falsehood because they tend to exaggerate the results of the investigation (Sumner et al., 2014).

The case of the news coverage of attention-deficit hyperactivity disorder illustrates this journalistic tendency to report on initial and uncertain results that eventually turn out to be false (Gononon, Konsman, Cohen, & Boraud, 2012).

Discussion and conclusions

Both pseudosciences and bad science offer uncertain or false knowledge about biomedicine in general and medical interventions in particular. The scientific community is aware of the danger posed by pseudosciences and is also beginning to realise that bad science is a threat to scientific knowledge; it represents a major waste of resources – 85 % of global research expenditure is estimated to be wasted on redundant or poorly designed studies (Chalmers & Glasziou, 2009) – and may also entail health risks.

Unlike pseudoscience, science is self-correcting and has launched studies and initiatives to discern the quality of evidence, mitigate the reproducibility crisis, and curb bad science. Although this movement has met with logical resistance, there are already proposals aimed at placing more emphasis on methodology and changing the incentive system for researchers, among other aspects (Munafò et al., 2017).

«While science is wary of newer studies, journalism seeks them out and pays more attention to them»

We need more studies on how pseudoscience and bad science affect the dissemination of scientific knowledge and the development of critical thinking. The studies available about potential health risks are also scarce and limited, especially in their description and explanation of the significance of bad science and low and very low-quality evidence in decision-making and making medical recommendations. There is also a lack of more complete and rigorous studies aimed at understanding the impact that pseudoscience and bad science can have on the media and on social networks, where both types of messages are mixed and confused. Analysing these three aspects, among others, would be interesting for theoretical and practical reasons, as both pseudoscience and bad science have a negative impact on the culture, economy, and well-being of citizens. In this context, comparative studies are also relevant, as there are indications that the problem of bad science may be more serious than the one posed by pseudoscience.