The challenge of clinical research

The best tool we have to make good clinical decisions is the scientific method, as it is free, critical and dynamic to test the corresponding hypothesis. The standards of the scientific method can and should be applied to all treatments and therapies with the necessary flexibility but, at the same time, with the same precision, regardless of their nature, characteristics or eventual and sometimes random classification into conventional, complementary or alternative.

The challenge to differentiate between good and bad treatments

It is essential to determine to what extent the benefits attributed to treatments are reliable enough, especially with life-saving, invasive or aggressive treatments, or very expensive ones. Even if patients do not suffer adverse effects after having less effective treatments, using them could mean a waste of both individual and collective resources. A few years ago the publication Clinical Evidence analysed the effectiveness of 3,000 treatments. Only 11% of these studies showed that the treatments were beneficial; 24%, that they probably were not; but the others had unknown, improbable or low efficacy.

Extreme variability

There is a wide variability in the use of certain treatments. The causes can be many, but among the most prominent ones we find the wrong and contradictory interpretation of available evidence, which is often insufficient. However, other times it is unknown, not well interpreted or badly applied. For example, a recent study by our group (Quintana, 2016) with data from all the Spanish hospitals has shown that the use of endovascular surgery to treat a condition as serious as abdominal aortic aneurysm varies between 3% and 78%.

The importance of the placebo effect

Many diseases tend to worsen without treatment and others get worse despite the treatments. However, some health problems get better on their own, i.e., they are self-limited, because human nature fights back most of the time and heals on its own. But, of course, a treatment could also make things worse. Some doctors and patients above all usually ascribe the observed improvements to the treatment, although it had nothing to do. Therefore, research has to be able to discern between the true effects of the treatments and those linked to nature itself or to the placebo effect (the psychological effect produced by an inert substance). We have recently been able to demonstrate following a controlled study (De la Poza et al., 2016) that not giving antibiotics immediately to treat mild respiratory infections is better than their systematic prescription, often given credit for healing self-limited diseases.

In addition, the rooted tradition over time in many social security centres leaves no room to even critically consider if there are better therapeutic alternatives than the ones in use. Individual clinical experience should not be the enemy of evidence generated through clinical studies carried out in other patients.

New is not always better

A culture based on optimistic and materialistic minds prevails in our society, and health care is no exception. This is the reason why novelties are presented as something better, frequently exaggerating the beneficial effects of the treatments and playing down the harmful ones. Medicine hides a background full of large and small health disasters caused by the premature and hasty introduction of new treatments without proper evaluation. The case of thalidomide, which caused severe congenital malformations to the children of the women treated with it while pregnant; or the case of the blindness epidemic among preterm newborn infants exposed to high oxygen concentrations are emblematic in that painful history.

More is not necessarily more beneficial

The search for the correct intensity – the one that offers extensive benefits and brings few adverse effects – is the common challenge among all treatments, but this kind of studies are unusual and the respective decisions are taken following subjective schemes. Often, as the dose or the duration is increased, beneficial effects find their limit, but – generally – adverse effects increase. Aspirin is an excellent example: a low dose helps to prevent strokes and has very few adverse effects, while the higher the dose, the higher the risk of suffering a haemorrhage.

Anticipation is not always positive

The goal of secondary prevention or the so-called early detection is to identify injuries or risk factors for some important diseases through a screening test. We have to necessarily wonder if the benefits following a certain screening test actually go beyond its risks and disadvantages. The contrast between the high increase of prostate cancer cases detected from the massive determination of the prostate-specific antigen and the lack of mortality reduction from this disease is a very illustrative example.

The role of industry

To carry out the corresponding clinical studies and – at the same time – having more credibility, proprietary companies owning the products need researchers to recruit the appropriate patients and later to spread their results. Whether these results are more or less reliable will depend on the influence of the conflicts of interests, called like this because patients interests – the one doctors are deontologically committed to respect – and the interests of the companies funding the research and the interests of the researchers themselves may be in conflict. Getting to the bottom of these connections between research and business is not easy, despite the increasing demand to state conflicts of interest explicitly when research studies are published. As a result, independent studies without conflicts of interest are still essential to help doctors fulfilling their role as «unbiased protectors of their patients’ interests and critical evaluators of new products and services.» (Relman, 1991).

The influence of chance

We need studies not only to try to minimise biases, but also studies that take into account the influence of chance, which may confuse the direction and magnitude of results when the therapeutic effects are small. Moreover, if these effects get to be evaluated through studies with a short programme, it will then be impossible to distinguish between the differences that can be attributed to treatments and those that are produced by mere chance.

The pressure of the affected people

When a specific popular opinion about the efficacy of a treatment is widespread – more often than not through manipulation – it is difficult to defend the need of a study comparing that treatment with a conventional one or a placebo. It is seen as an attempt to deny access to an effective therapy and save money. In some serious diseases, patients have not agreed to take part in the corresponding studies because they simply wanted to have the potentially miraculous treatment right away. In many complementary therapies, such as homeopathy, the users are so convinced about their effectiveness that they would hardly agree to expose themselves to an independent evaluation.

Not prioritising research

Clinical research questions should arise from the uncertainties and limitations detected in the clinical practice itself, with the direct participation of patients, if possible. But in many cases the industry is very influential in the decision-making processes regarding what issues will be researched. Accordingly, many research studies with low clinical priority are boosted; a large part of the new treatments offer small variations in comparison to the others above; interventions without commercial value are hardly ever studied, such as diet advice, exercise, physiotherapy or end-of-life care, which are only a few different examples.

Redundant or unnecessary research

Many therapeutic clinical studies are redundant and, therefore, unnecessary, regardless of their quality. Such works do not provide virtually anything new at best and, at worst, only bring confusion and distortion to the patients. One of the major flaws of these studies is that they are designed without consulting and taking into account prior evidence about the issue, so the same mistakes and limitations detected by previous authors appear again.

El millor disseny d’un assaig clínic controlat és el de distribució aleatòria, un sistema que ben emprat crea grups d’individus perfectament comparables. D’aquesta manera, es poden atribuir les possibles diferències observades als diferents tractaments rebuts per cadascun. / Edu Bayer/SINC

Comparisons, the key to therapeutic studies

The need for contrast between two groups of patients with different treatments has existed in medicine for a long time. However, the single systematic difference between both patients groups should be their respective treatments, so any possible differences observed in the effects are reasonably attributed to them, preventing them from being determined by the irregular prognosis of the individuals included in the compared groups. Comparisons that apply historical controls are potentially problematic, because we can always doubt whether the compared groups are similar or not. Nevertheless, if the controls are contemporary and not historical the challenge is still there, now in knowing whether the patient groups undergoing the different interventions are similar enough or not before beginning the treatment. Therefore, the results of observational studies used to solve therapeutic questions must be always received very cautiously.

Hormone replacement therapy is a clear example of this situation, a therapy followed for many years to prevent cardiovascular disease according to observational studies, but later dismissed after the clinical trials detected the potentially severe adverse effects.

To achieve a neutral and impartial designation of individuals for the different treatments, randomised controlled clinical trial are the best method. When the randomisation system is properly applied, it creates perfectly comparable groups of individuals. It is possible then to attribute the observed results reasonably to the different treatments that the intervention group and the control one have.

However, trials also have a range of potential methodological and logistical problems that, due to their importance, might completely supersede their relevance and credibility: excessively restrictive selection criteria, incorrect designation, deviations from the initial protocol, insufficient study size, early interruption of studies, subjective selection or analysis of the results, abuse of compound variables, little consideration of the adverse effects and excessive bureaucratisation of the procedures, among others.

Systematic reviews, to the rescue

Many of the problems of primary controlled clinical studies can be solved – or at least controlled – with a proper review of the literature. In fact, checking the medical literature that has tried to answer a particular therapeutic question necessarily involves going beyond a single primary study and locate all the suitable works that have addressed the same issue. This way, the evidence provided by a particular study is situated in the context of existing knowledge and, therefore, an extensive and rigorous review involves a global perspective – and at the same time specific – that makes it essential (Higgins and Green, 2011).

A review must be systematic to integrate all the empirical evidence and comply with some previously established eligibility criteria. It uses rigorous and explicit methods with the aim of minimising the biases and thus providing more reliable results from which to draw conclusions and make decisions.

The importance and growing prestige that systematic reviews have acquired are mostly due to the impulse from the Cochrane collaboration (Bonfill, 2014). It is an independent worldwide network of health professionals, as well as citizens and patients who have as their main goal producing, updating and disseminating systematic reviews of clinical and health interventions. Cochrane reviews, due to their higher methodological quality and wide dissemination, have become the reference across the world.

Nevertheless, systematic reviews are not a silver bullet, because they also face a range of challenges and problems that are not small and should be stressed, such as low-quality primary research, bias in publication, inadequate review methods or redundancy in a large part of the published reviews.

Future perspective of the clinical research

Everything we have seen so far leads us to the need for promoting ethical values in therapeutic clinical research and establishing the mechanisms at every level to apply them to praxis. It is also necessary to add conceptual, methodological and logistical improvements so the information derived from clinical research will be more appropriate.

From the methodological point of view, during the last decades we have learnt different ways through which the reliability of the clinical research can be weakened. But at the same time, a lot of different methods have been developed to prevent many errors and traps, both conscious and unconscious. In addition, it has to be possible to combine the essential structure of clinical trials with the potential of the markers that the basic research continuously provides. We are talking about innovation in the so-called adaptive designs, in which the researchers in a trial identify the results of a group of patients who have already undergone a treatment to decide how future patients included in the trial will be treated.

It is also important to take all the potential of the observational studies without random distribution, that is, those studies with a control group not formed by chance. Observational studies are not only useful to detect the adverse effects of treatments that appear in the medium and long term or to evaluate the results of the treatment when a certain local or territorial level is added. It is also possible and necessary to use them with precision when carrying out a clinical trial is not possible or a advisable. For example, with rare diseases or when the expected therapeutic effects are significant.

Besides improving primary research, there are also other specific challenges for systematic reviews ; among others: recording review protocols prospectively in open access platforms; developing technological support tools to link primary studies to systematic reviews and make the review process more efficient; assuming and standardising the combination of the results from clinical trials and observational studies better, both with regards to adverse effects and to the costs linked to effectiveness, when relevant.

There is no doubt that therapeutic clinical research is essential in order to improve people health through the creation of useful information about its possible benefits and adverse effects. However, empirical studies suggest that most of the therapeutic effects found in medical research are falsely positive or have been fundamentally exaggerated, so an excessive number of promising scientific advances fail to meet the expectations. We need more public resources to boost quality independent clinical research but, at the same time, it is necessary to minimise the kind that only leads to confusion and delay in the application of well-based clinical decisions.

[This article is a summary of the keynote speech made by Xavier Bonfill as a full member of the Biological Sciences Section of the Institute of Catalan Studies.]