Bayesian inference

• 1. Introduction

  ›

  Bayesian inference begins with a single, disarming idea: your beliefs are probabilities, and every new piece of evidence should change them. The method takes its name from Thomas Bayes, an 18th-century English minister who died in 1761, leaving behind an essay that would quietly reshape how scientists, doctors, engineers, and courts think about uncertainty. Pierre-Simon Laplace, working decades later, formalized what we now call Bayes' theorem and applied it to problems in celestial mechanics, medical statistics, and jurisprudence. The theorem sat on the margins of mainstream science for generations before exploding back into relevance in the 1980s. What drove that revival, how the method actually works, and why a courtroom in the United Kingdom once grappled with whether to teach a jury the formula, are the questions this documentary sets out to answer.

• 2. The Logic of Updating

  ›

  Bayes' theorem rests on a deceptively compact relationship between three quantities: a prior probability, a likelihood, and a posterior probability. The prior is what you believed before seeing any evidence. The likelihood measures how well the observed evidence fits a given hypothesis. The posterior is what you believe after combining both, and it becomes the new prior when fresh evidence arrives. This cycle of updating is the engine of Bayesian inference. The rule can be stated plainly: the posterior is proportional to the likelihood multiplied by the prior. The marginal likelihood, sometimes called the model evidence, normalizes the result so that all competing hypotheses still add up to one. One property of the theorem carries a sharp warning. If you assign a hypothesis a prior probability of exactly zero, no amount of evidence can ever raise it above zero. Statistician Ian Hacking pointed out that traditional Dutch book arguments did not actually require Bayesian updating; they left open the possibility that other updating rules could avoid the same pitfalls. Richard C. Jeffrey later developed an updating rule for cases where the evidence itself is uncertain, a framework discussed under the name probability kinematics.

• 3. Cromwell and Convergence

  ›

  A principle sometimes called Cromwell's rule captures the hard edge of the theorem: if a prior probability is set to exactly zero or exactly one, it can never be moved by evidence. This is not a flaw but a logical consequence, and it counsels against certainty before data. When prior probabilities are less extreme, Bayesian inference has better news. Under conditions first rigorously worked out by Joseph L. Doob in 1948, the Bernstein-von Mises theorem guarantees that as the number of observations grows, the posterior distribution converges to a Gaussian centered on the true value, regardless of where you started. David A. Freedman extended this work in research papers published in 1963 and 1965. His 1963 paper confirmed convergence for finite probability spaces. His 1965 paper demonstrated that for a countably infinite probability space, a dense subset of priors fails to converge at all. Freedman and Persi Diaconis continued to probe these infinite cases through the 1980s and 1990s, and the practical lesson remains: for large but finite systems, convergence may be very slow, and the initial choice of prior can matter for a long time.

• 4. Inference in Action

  ›

  A bowl of cookies clarifies the abstract machinery. Two bowls sit on a table: bowl one holds 10 chocolate chip and 30 plain cookies, bowl two holds 20 of each. Fred picks a bowl at random and draws a plain cookie. Bayesian inference asks: given the plain cookie, how likely is it that Fred reached into bowl one? Intuition says more than half, because bowl one is heavier in plain cookies. Bayes' theorem makes that precise. Since each bowl was equally likely before the draw, and a plain cookie is more probable from bowl one, the calculation returns a posterior probability of 0.6 for bowl one. An archaeological example runs the same logic over a longer time span. A site believed to be medieval, ranging from the 11th through the 16th century, yields pottery fragments that are either glazed or decorated. If inhabited early, 1% of pottery would be glazed and 50% decorated; if late, 81% glazed and 5% decorated. As each new fragment is unearthed, the archaeologist's belief shifts. A simulation of 50 fragments, anchored to an assumed true occupation date of around 1420, produced a posterior giving the site about a 63% chance of 14th-century occupation and about 36% for the 15th century, with practically no probability for the 11th or 12th centuries.

• 5. Prediction and Its Rivals

  ›

  Bayesian prediction differs from frequentist prediction in a consequential way. Rather than fixing a single best-guess parameter and plugging it into a formula, the Bayesian approach integrates over all possible parameter values, weighted by the posterior. The result is the posterior predictive distribution, a full spread of possible outcomes rather than a point. Frequentist methods sometimes correct for parameter uncertainty in special cases, the Student's t-distribution being the classic example for normally distributed data with unknown mean and variance. Bayesian methods, by contrast, can always determine this predictive spread exactly, or to any desired level of precision using numerical methods. Abraham Wald gave decision theory a landmark result: every unique Bayesian procedure is admissible, and every admissible statistical procedure is either a Bayesian one or a limit of Bayesian procedures. This placed Bayesian methods at the core of frequentist inference tasks such as parameter estimation, hypothesis testing, and confidence intervals, even for practitioners who did not think of themselves as Bayesians.

• 6. Bayesian Inference on Trial

  ›

  In the United Kingdom, a case called R v Adams tested whether Bayes' theorem belonged in a criminal courtroom. A defense expert explained the theorem to the jury. The jury convicted, and the case went to appeal on the grounds that no mechanism had been provided for jurors who declined to use the formula. The Court of Appeal upheld the conviction but issued a pointed opinion: introducing Bayes' theorem, or any similar method, into a criminal trial plunges the jury into inappropriate and unnecessary realms of theory and complexity, deflecting them from their proper task. The controversy surfaces a real tension. Bayes applied to a criminal trial requires a prior probability of guilt. One suggestion in the literature is that if a thousand people could plausibly have committed a crime, the prior should be set at 1/1000. A scholar named Gardner-Medwin argued that the jury's criterion should not be the probability of guilt but the probability that the observed evidence could have arisen if the defendant were innocent, a framing closer to the frequentist concept of a p-value. He proposed that conviction should require believing both that the evidence fits a guilty defendant and that it could not have arisen from an innocent one. The practical suggestion that emerged from these debates is that Bayes' theorem might be more accessible to jurors when expressed as betting odds rather than probabilities, since odds are a more familiar language.

• 7. The Computing Revolution

  ›

  Bayesian methods spent much of the 20th century on the sidelines because the computations were intractable for all but the simplest models. The 1980s changed that. The discovery of Markov chain Monte Carlo methods removed most of the computational barriers, enabling Bayesian analysis of models whose posterior distributions had no closed-form solution. Gibbs sampling and the broader family of Metropolis-Hastings schemes are named examples of these simulation-based techniques. Probabilistic programming languages emerged as a practical layer on top of these methods, letting practitioners build Bayesian models while leaving the numerical inference to automated tools. The connection between Bayesian methods and machine learning deepened steadily, and spam filtering became an early, widely noticed application. Filters built on Bayesian inference, including CRM114, DSPAM, Bogofilter, SpamAssassin, SpamBayes, and XEAMS, learned to distinguish unwanted messages by treating the probability that a message is spam as a hypothesis to be updated on each word. Pattern recognition work using these techniques dates back to the late 1950s.

• 8. Science at the Edge

  ›

  Cosmology adopted Bayesian methods for a task with no frequentist alternative: fitting the six base parameters of the Lambda-CDM model to data from the cosmic microwave background. The Planck 2018 CMB dataset is a cited example of this approach in practice. The software package cobaya sets up these cosmological runs, interfacing with Boltzmann codes that compute predicted CMB anisotropies for any trial set of parameters, and driving MCMC or nested samplers to explore the posterior. The same machinery applies to extended theories of cosmology, including models with early dark energy or modified gravity, where Bayesian model comparison can weigh the evidence for a non-standard model against the baseline Lambda-CDM. In biology and medicine, Bayesian methods power differential gene expression analysis and a cancer-risk framework called CIRI, the Continuous Individualized Risk Index, which incorporates serial measurements to update a patient-specific model built from prior clinical knowledge. In astrophysics, the method has been used to characterize the atmosphere of the exoplanet k2-18b. The phylogenetics community adopted Bayesian inference because it allows simultaneous estimation of many demographic and evolutionary parameters, a task that would be far more cumbersome by other means. Thomas Bayes wrote his foundational essay on the binomial distribution's success rate, now called Proposition 9 of his work; the application domains that proposition eventually licensed now span from the subatomic to the cosmological.