Scientific control

• 1. Introduction

  ›

  Scientific control sits at the heart of every trustworthy experiment, yet most people have never thought much about what it actually is. Imagine a researcher who feeds an artificial sweetener to sixty laboratory rats and watches ten of them fall ill. Is the sweetener the culprit? Or was it something in the dilutant used to mix it? Without a control, there is no way to know. That uncertainty is exactly what scientific controls are designed to dissolve.

  A control is an element built into an experiment specifically to shrink the influence of variables other than the one being studied. The goal is to give researchers a clean baseline, a reference point against which every measurement can be compared. Without that baseline, results may look meaningful when they are actually noise.

  Controls are not a single tool but a family of them: negative controls, positive controls, randomization, blinding, sham procedures, and more. Each addresses a different way an experiment can go wrong. And the challenge of choosing the right ones, as researchers in biology, chemistry, medicine, and psychology know well, can be surprisingly difficult. The question of which controls to use, and how to use them, is where the real craft of experimental design lives.

• 2. Confounding and Its Costs

  ›

  Confounding is the reason controls exist in the first place. A confounder is an outside variable that is connected to both the thing being studied and the outcome being measured, quietly distorting what the data appear to show. When a study fails to account for one, the results can point researchers toward completely wrong conclusions.

  Consider a study that sets out to understand the relationship between physical activity and heart disease. If researchers do not control for diet, a powerful confounder, they risk overestimating or underestimating how much exercise actually matters. The diet is shaping the outcome alongside the exercise, but without a control it remains invisible in the analysis.

  The consequences reach beyond the lab. Uncontrolled confounding can produce incorrect policy recommendations, launch ineffective medical interventions, and leave behind a flawed body of scientific understanding that takes years to correct. This is why fields like epidemiology and social science have put growing effort into developing tools specifically designed to detect confounding after the fact, when the cleaner path of a randomized experiment is not available.

  Falsification tests are one such tool. Researchers apply their analytical method to a scenario where no effect should exist. If an association still appears where none belongs, that is a signal that confounding or bias may be distorting the primary analysis. Negative controls are a specific type of falsification test, and their use in the epidemiology literature has grown steadily, with researchers now exploring their application in economics as well.

• 3. Negative Controls in Practice

  ›

  Lousdal and colleagues ran a study examining whether screening participation affected death from breast cancer. Their hypothesis was that people who sign up for screening are generally healthier than those who do not, meaning a raw comparison of mortality rates between the two groups would be misleading from the start. To test whether that healthy-user bias was distorting their results, they selected negative control exposures and negative control outcomes as diagnostic tools.

  Death from causes unrelated to breast cancer served as a negative control outcome. Since breast cancer screening cannot plausibly prevent death from unrelated diseases, any difference in that mortality rate between screeners and non-screeners would have to come from the underlying health gap between the groups. Dental care participation was chosen as a negative control exposure, taken as a marker of health-attentive behavior rather than a cause of any health outcome.

  Yerushalmy's work on smoking offers a different illustration. The exposure under study was maternal smoking during pregnancy; the outcomes included low birth weight, length of pregnancy, and neonatal mortality rates. Yerushalmy used the husband's smoking as a negative control exposure. The logic was that a husband's smoking shares household lifestyle confounders with the mother's smoking but cannot directly affect fetal development. Yet a statistical association appeared anyway, casting doubt on whether maternal smoking was truly the causal agent in the birth outcomes observed.

  Jackson and colleagues applied the same logic to influenza vaccines, using mortality from all causes outside the influenza season as a negative control outcome. Their analysis found that healthier seniors were more likely to receive the vaccine, and that these health differences between vaccinated and unvaccinated groups were introducing bias into estimates of vaccine effectiveness. Sheppard and colleagues used non-elderly appendicitis hospital admissions as a negative control outcome when studying whether air pollutants drove asthma hospitalizations. The thread connecting all these examples is the same: find an outcome or exposure that shares the confounding mechanism but lacks any plausible causal link to the treatment, and see whether an association shows up regardless.

• 4. Formal Rules for Negative Controls

  ›

  Shi and colleagues set out to give negative control outcomes a rigorous mathematical foundation, specifying the conditions that must hold for an NCO to be valid. Four assumptions form the core of their framework.

  The first is the Stable Unit Treatment Value Assumption, which governs how treatments interact across units. The second, Latent Exchangeability, requires that a subject's potential outcome be independent of the treatment once the control variables and unmeasured confounders are taken into account. A violation arises, for example, if only the people who will actually benefit from a medicine tend to take it, even when age and medical history are held constant. The third assumption, Irrelevancy, requires that the treatment have no causal effect on the negative control outcome. In the influenza vaccine context, this means the vaccine itself should not reduce all-cause mortality. But if a physician administering the vaccine also performs a general physical exam, recommends healthy habits, and prescribes vitamins, then the medical visit may be doing work beyond the vaccine itself, potentially breaking the Irrelevancy condition.

  The fourth condition is U-Comparability: the unmeasured confounders driving the association between the treatment and the outcome must be the same confounders driving the association between the treatment and the NCO. If a researcher picks an NCO that is only weakly correlated with the unmeasured confounders, U-Comparability is violated and the test loses its diagnostic power. Even when all four conditions hold, a non-null association between the treatment and the NCO points toward the unmeasured confounder as the explanation, rather than any direct causal path. An important limit also applies: a negative control test can reject a study design, but it cannot by itself validate one.

• 5. Positive Controls and Calibration

  ›

  Positive controls serve a purpose that is almost the mirror image of their negative counterparts. Where a negative control checks for spurious effects, a positive control checks that an experiment is capable of detecting real ones. A test that cannot find a known effect has no business looking for unknown ones.

  In an enzyme assay designed to measure the amount of an enzyme in a set of extracts, a positive control would consist of a sample containing a precisely known quantity of the purified enzyme. That sample should produce a large, clear enzyme activity reading. If it does not, something is wrong with the experimental procedure itself, and the experiment needs to be repeated before any other conclusions can be drawn.

  When diagnosing a disease using a new test, the positive control is an older, well-established test already known to work. Running the new test alongside the established one lets researchers verify that the new test functions correctly by checking whether both tests agree on cases where the answer is already known.

  Multiple positive controls offer an additional benefit: calibration. When two or more known-effective tests are included, and each is expected to produce a result of a different size, researchers can construct a standard curve. In the enzyme assay example, preparing many samples with different known quantities of the enzyme generates a curve that allows fine-grained comparisons and standardization across experiments. If the established test produces the same result that previous experimenters found, it confirms that the current experiment is being conducted the same way as those earlier ones, lending confidence to any comparisons drawn across studies.

• 6. Randomization and Blinding

  ›

  Randomization addresses a different category of problem. When groups in an experiment are assigned treatments by chance, any differences between them are distributed equally across conditions rather than clustering in one group. This does not eliminate all differences between groups, but it prevents those differences from systematically favoring or working against any particular treatment. In crop yield experiments where soil fertility varies across plots, randomly assigning which plots receive which treatment keeps variation in soil composition from skewing the results in one direction.

  Blinding addresses the problem of expectation. When participants, researchers, or evaluators know who received which treatment, that knowledge can reshape the experiment in ways that have nothing to do with the treatment itself. Patients who know they received an active drug may experience a stronger placebo effect. Researchers who know which group received the intervention may, consciously or not, handle the groups differently, an instance of the observer effect. Evaluators who know the treatment assignment may rate outcomes in ways that reflect confirmation bias rather than the data in front of them. Blinding can be applied to any of these parties, and in some medical trials, sham surgery may be necessary to make blinding possible.

  Unblinding, when a participant deduces or otherwise learns information that was meant to be hidden, is a recognized source of experimental error. Once the concealed information surfaces, the bias that blinding was meant to suppress re-enters the experiment. Meta-research has documented high levels of unblinding in pharmacological trials, with antidepressant trials identified as particularly poorly blinded. Reporting guidelines call for all studies to assess and report on unblinding, but in practice very few studies actually do. A trial that operates with no blinding at all is called an open trial in clinical research.