That’s right, earlier today, the Cochrane Collaboration announced that it has “formalized its commitment to the AllTrials: All Trials Registered | All Results Reported initiative.”
This is great news, but I’m honestly a bit surprised that it took them so long. The Cochrane Collaboration is currently celebrating its 20th anniversary as a leader in the movement for evidence-based medicine. It was started as an answer to Archie Cochrane’s (a British epidemiologist) calls to maintain up-to-date, systematic reviews of randomized controlled trials (RCTs) of medical interventions. Cochrane Reviews, as they are called, are the gold standard in systematic reviews in the medical literature. This is because they use rigorous, standardized methods, published in the Cochrane Handbook, to ensure that the reviews are consistent, transparent, comprehensive, and as objective as possible. They are also accessible for free in over 100 low and middle-income countries (LMICs) (too bad they’re not open access).
When I first heard of Cochrane a few years ago, I couldn’t believe that something like them wasn’t already standard practice across the board. Come to think of it, I felt the same way when I first heard of “evidence-based medicine” for the first time. Wait, you mean huge portions of medical practice (even if you don’t count so-called “alternative” “medical” woo like homeopathy) are not based on evidence? Yes, it’s unfortunately true, or at least it was true before evidence-based medicine began to gather steam in the last two decades. Take, for example, the Cochrane logo:
The logo is what is called a forest plot[1] and this one is a tragic reminder of the danger of not using systematic reviews. Each horizontal line shows the results of one study. The vertical line is the point where the treatment does not help people, but it doesn’t hurt them either. The left side of the vertical line means that the treatment is good and the right side means that it’s bad. The length of the line shows how precise the results are (the shorter the line, the more precise it is). If a line touches the vertical line, that means the treatment didn’t make a clear difference. The diamond at the bottom is what you get when you combine all the studies. In this case, it’s for giving hormones called corticosteroids to pregnant women who are going to give birth too early in order to prevent their babies from dying. The study at the top was done in 1972.[2] Ten years later, it should have been clear that the corticosteroids worked (we now know they reduce the odds of babies dying by 30% to 50%). But, a systematic review was not published until 1989, so many doctors did not know how good corticosteroids were. Because of that, tens of thousands of babies probably died.
It’s not an exact science, and it takes a lot of hard work to develop systematic reviews, but they really do save lives and prevent needless suffering. However, even though Cochrane has the world’s largest database of clinical trials, it is still vulnerable to the effects of not reporting the results of trials, or of not even registering them in the first place. I’m glad to see them join the campaign to get all trials registered and all results reported.
1. Though the plots themselves were being used by at least the 1970s, the term “forest plot” itself was not used in print until 1996 in Pittsburgh in a poster presented at a meeting of the Society for Clinical Trials in May 1996.
2. The 1972 study looks like it should have been clear that corticosteroids are awesome after just one study, but it’s risky to trust just one study. Science works because we’re always testing each other’s theories and questioning our results. If no one else can get the same results, then we throw them out and try something else.
Secular Trends 