Evidence Based MedicineIntensive CareMedical EducationSMACCTrial Design is the biggest problem with Evidenced Based Medicine

Trial Design is the biggest problem with Evidence Based Medicine by Paul Young.

Paul Young wants to change that paradigm completely. He argues for research as we know it to change and to focus on clinical care with systemised and optimised treatments that reliably improves outcomes over time for all patients.

Mortality measured at a particular time point (landmark mortality) is often regarded as the gold standard outcome for randomised controlled trials in Intensive Care Medicine. An important limitation of many Intensive Care Medicine trials is that they hypothesise large and potentially implausible reductions in absolute mortality. This is a major problem in trial design for two reasons.

Firstly, it makes false negative trial results more likely.

Secondly, the less plausible a postulated mortality reduction is the more likely that a statistically significant mortality difference will represent a false positive.

This is because a p-value is defined as the probability of finding a result equal to or more extreme than what is observed. This is under the assumption that the null hypothesis is true.

This means that the greater the pre-trial chance that the null hypothesis is correct, the lower the chance that a p-value below a particular significance threshold will represent a true positive.

In Paul’s words, p-values suck! The biggest single problem with the current evidence base is that most hypotheses being tested have low prior probability.

This leads to the two most likely results being:

1) no difference

or 2) a false positive.

We need a new research paradigm to address this problem, particularly in relation to the fundamentals of Intensive Care Medicine.

Moreover, Paul argues for an approach based on the Bayesian approach, utilising big data sources about patients under current care and then randomising treatment in real time. Intensive Care therapy is fundamentally about providing supportive care. Including airway support, oxygen therapy, ventilation therapy and haemodynamic support amongst others. These treatments can be uncertain.

Under such conditions of uncertainty and idiosyncratic practice variation, treatment should be randomised. Randomised treatment is likely to be the best treatment is these situations due to inherent cognitive biases.

Using this data, Paul stipulates that we will rapidly learn from every patient, ensuring improved outcomes for each subsequent patient. As knowledge grows, prior probability can be adjusted, skewing the randomisation process. Furthermore, this continues until the treatment has either proven effective, or shown to be equivalent.

Every patient contributes information that enhances the care of every subsequent patient. And, importantly, p-values no longer matter. Effective research is important. As Paul states, “Optimising ICU care is a priority for global public health.”

Evidently, by increasing the probability that patients will get the treatments that work even before we know what those treatments are the Intensive Care Unit can not only save money, but most importantly and critically, save lives. That is the way research should work.

Trial Design is the biggest problem with Evidence Based Medicine by Paul Young.

Finally, for more like this, head to our podcast page. #CodaPodcast

Paul Young

Paul Young is a member of the Australian and New Zealand Intensive Care Society Clinical Trials Group (ANZICS CTG). His primary interest is in the design and conduct of large-scale multicentre clinical trials in the field of Intensive Care Medicine.He is the co-clinical leader and director of research at Wellington ICU, the clinical director at Wakefield Hospital ICU, and the ICU programme director at the Medical Research Institute of New Zealand.

@DogICUma