Randomized controlled trial

• 1. The Backpack And The Parachute

  ›

  Two Boston-area medical researchers handed each of 23 volunteers either a parachute or an empty backpack, then asked them to jump from a biplane or a helicopter. The study reported its finding with a straight face. Parachutes failed to reduce injury compared to empty backpacks. The catch, left for the careful reader to notice, was that both aircraft were parked on the ground and the volunteers fell about two feet. This deadpan experiment was a warning dressed as data. It was making fun of a method that the rest of medicine treats as the gold standard, the randomized controlled trial. An RCT is a statistical experiment built to test whether a drug, a surgery, a device or a diet actually works. It does this by tossing a kind of coin. Participants are sorted at random into groups, some receiving the intervention and others receiving a placebo, a competing treatment, or standard care. How did a coin toss come to sit at the top of medical evidence? Who first imagined randomizing sick people? And why do critics keep poking holes in the very tool they trust most?

• 2. The Logic Of The Coin

  ›

  Random allocation is the whole trick, and it solves a problem that careful observation cannot. People who enroll in a trial differ from one another in known and unknown ways, and those differences quietly shape outcomes. By assigning participants to treatments at random, an RCT balances both known and unknown prognostic factors across the groups. This is how it reduces selection bias and allocation bias. Probability theory then does its part. Randomization lets researchers express the likelihood that any gap in outcome between groups is merely chance. Blinding is the second pillar. Participants are not told which treatment they received, and the same masking ideally extends to researchers, technicians, data analysts and evaluators. Effective blinding separates the real physiological effect of a treatment from the psychological sources of bias that cluster around expectation. There is a vocabulary trap worth flagging. The terms RCT and randomized trial get used as if they mean the same thing, but a randomized trial may lack a control group entirely. The initialism also stretches into randomized clinical trial and randomized comparative trial, which muddies the literature. One more caution lives inside the method. Empirical evidence that adequate randomization actually changes outcomes, compared with inadequate randomization, has been difficult to detect.

• 3. From Bloodletting To Streptomycin

  ›

  In the posthumously published Ortus Medicinae of 1648, Jan Baptist van Helmont sketched the first proposal for an RCT, pitting two ways of treating fever against each other. One arm would use Galenic medicine with its bloodletting and purging. The other would use van Helmont's own approach. He likely never ran it. James Lind conducted the first reported clinical trial in 1747, hunting a cure for scurvy, and the Irish physician James Henry elaborated principles for controlled trials in 1843. Blinding has its own lineage. The French Royal Commission on Animal Magnetism ran the first blind experiment in 1784 to test the claims of mesmerism. Later in the 19th century, Claude Bernard argued that the observer of an experiment should not know the hypothesis being tested, a stance that clashed with the Enlightenment faith that only a well-educated scientist could observe objectively. The first recorded study with a blinded researcher came in 1907, when W. H. R. Rivers and H. N. Webber probed the effects of caffeine. Randomization itself drifted in from other fields. Charles Sanders Peirce and Joseph Jastrow introduced randomized experiments in psychology in the 1880s. Robert Woodworth and Edward Thorndike published early treatment-versus-control work in 1901, followed by John E. Coover and Frank Angell in 1907. In agriculture, Jerzy Neyman and Ronald A. Fisher carried the idea forward, and Fisher's writings popularized it. The threads converged in 1948. A paper titled Streptomycin treatment of pulmonary tuberculosis, describing a Medical Research Council investigation, became the first published RCT in medicine. One of its authors was Austin Bradford Hill, credited with conceiving the modern RCT. By the late 20th century the method was the recognized standard for rational therapeutics. As of 2004, more than 150,000 RCTs sat in the Cochrane Library.

• 4. Many Shapes Of A Trial

  ›

  Parallel-group is the workhorse design, the one most reporting standards assume. Each participant is assigned to a single group and everyone in that group either gets the intervention or does not. An analysis of 616 RCTs indexed in PubMed during December 2006 found that 78% were parallel-group, with crossover at 16% and split-body, cluster and factorial each at 2%. The alternatives bend that basic shape. In a crossover trial each participant receives the intervention, or not, in a random sequence over time. A stepped-wedge trial moves clusters of subjects from control to intervention in a random and sequential order until every cluster is exposed, a design once called a waiting list design. Cluster trials randomize pre-existing groups such as villages or schools. Factorial trials assign combinations, so one group might get vitamin X and vitamin Y while another gets vitamin X and a placebo. There are other ways to slice the category. By outcome of interest, an explanatory RCT tests efficacy under highly controlled conditions with carefully selected participants, while a pragmatic RCT tests effectiveness in everyday practice with relatively unselected participants. By hypothesis, most are superiority trials seeking to show one treatment beats another, while noninferiority trials ask whether a new treatment is no worse than a reference, and equivalence trials ask whether two are indistinguishable.

• 5. Guarding The Envelope

  ›

  Stories abound of investigators holding sealed envelopes up to the light, or ransacking offices, to learn which group their next patient would land in. That behavior is exactly what allocation concealment exists to defeat. Concealment protects the randomization process so the assigned treatment stays hidden until the patient is definitively entered into the study. Standard safeguards include sequentially numbered, opaque, sealed envelopes known as SNOSE, sequentially numbered containers, pharmacy controlled randomization, and central randomization. Generating the sequence is its own craft. Simple randomization works like repeated fair coin-tossing and resists both selection and accidental bias, but it can produce lopsided group sizes, so it is recommended only for trials of more than 200 subjects. Restricted methods exist to keep groups balanced in smaller trials. Permuted-block randomization fixes a block size and an allocation ratio, so a block of 6 at a 2:1 ratio sends 4 subjects one way and 2 the other. Adaptive biased-coin methods, of which urn randomization is best known, lower a group's odds when it is overrepresented and raise them when it is underrepresented. Rarer still are covariate-adaptive methods like minimization, where only the first subject's assignment is truly random, and response-adaptive methods that raise a group's odds when earlier patients in it fared well. The stakes of concealment showed up in the numbers. A 2008 study of 146 meta-analyses found that trials with inadequate or unclear allocation concealment were biased toward beneficial effects, but only when the outcomes were subjective rather than objective.

• 6. When The Method Backfires

  ›

  After Food and Drug Administration approval, the antiarrhythmic drugs flecainide and encainide reached the market in 1986 and 1987. The non-randomized studies behind them were described as glowing, and by early 1989 the two drugs accounted for roughly 165,000 prescriptions per month. That same year a preliminary RCT report concluded the drugs increased mortality, and sales fell. Hormone replacement therapy followed a similar arc. Before 2002, on the strength of observational studies, physicians routinely prescribed it to post-menopausal women to prevent heart attacks. Then RCTs from the Women's Health Initiative, published in 2002 and 2004, found that women taking estrogen plus progestin had a higher rate of myocardial infarctions than women on placebo, and that estrogen-only therapy did not reduce coronary heart disease. Use of the therapy declined afterward. These reversals help explain why scientific bodies rank RCTs so high. Australia's National Health and Medical Research Council, as of 1998, called a systematic review of all relevant RCTs Level I evidence. The GRADE Working Group concluded in 2008 that randomized trials without important limitations constitute high quality evidence. The Oxford Centre for Evidence-based Medicine, as of 2011, defined a consistent systematic review of RCTs as Level 1a. Yet the ranking is not absolute. Two studies in The New England Journal of Medicine in 2000 found that observational studies and RCTs overall produced similar results, and a 2014 Cochrane review, updated in 2024, found little evidence of significant effect differences between the two designs.

• 7. The Price Of Certainty

  ›

  One study counted 28 Phase III RCTs funded by the National Institute of Neurological Disorders and Stroke before 2000, costing US$335 million in total, an average of US$12 million each. The same study estimated those trials returned a net benefit to society at 10-years of 46 times their cost. Money is not the only burden. An RCT can take years to publish, so its data stays locked away from the medical community and may be less relevant by the time it appears. Rare events make matters worse. Studying something as infrequent as sudden infant death syndrome, or an uncommon drug side effect, would demand enormous sample sizes, which is why observational studies sometimes fit better. Ethics complicate the picture further. Placebo-controlled trials are deemed unethical when withholding treatment could harm a patient through worsening symptoms or risk of death. Active-controlled trials raise questions about clinical equipoise, the genuine uncertainty within the expert medical community about which treatment is preferred. Studies since 1982 have documented therapeutic misconception, where subjects believe they are certain to receive the treatment best for them and do not grasp the difference between research and treatment. Funding can tilt the scale too. A systematic review published in 2003 found that across four articles from 1986 to 2002, industry sponsorship correlated with positive study outcomes. A 2011 review of 29 meta-analyses, covering 509 RCTs, found that 219 of the 318 trials reporting funding sources, about 69%, were industry funded, yet only two of the meta-analyses reported those funding sources at all.

• 8. Beyond The Clinic

  ›

  In 1994 Paul Glewwe, an eventual Nobel Prize winner, together with Michael Kremer and Sylvie Moulin, launched one of the earliest RCTs in an economic setting, a long-run intervention in a school in Kenya, with results published fifteen years later. Three years on, in 1997, the PROGRESA program in Mexico became the largest field experiment in a developing context and drew a crowd of economic researchers. The method spread fast. The American Economic Association founded a registry of active and completed RCTs in 2013, and it has since tracked over 7,400 field experiments across 100 countries, with annual registries growing year over year. John A. List, a proponent of field experiments, argues that RCTs differ sharply from lab experiments and yield more robust identification. Other disciplines borrowed the tool with mixed enthusiasm. A 2005 review found 83 randomized experiments in criminology published between 1982 and 2004, up from 35 in the prior quarter century, sorted into policing, prevention, corrections, court and community. Between 1980 and 2016, education researchers published over 1,000 reports of RCTs, including a 2009 study that randomized 260 elementary school teachers' classrooms and another that followed 678 first-grade children through age 19. Not everyone is convinced the method travels well. In transport science, Dr. Steve Melia argued that the advantages of RCTs in establishing causality and avoiding bias have been exaggerated, and he proposed eight criteria for when behavior-changing interventions can fairly be tested this way. His first criterion narrows the field at once. The intervention must not have already been applied to every member of a unique group, such as the entire population of a country.