— Ch. 1 · Foundations And Definitions —
Bayesian network.
~5 min read · Ch. 1 of 6
A Bayesian network is a probabilistic graphical model that represents a set of variables and their conditional dependencies via a directed acyclic graph. This structure uses nodes to represent variables in the Bayesian sense, which may be observable quantities, latent variables, unknown parameters or hypotheses. Each edge in the graph represents a direct conditional dependency between two variables. Any pair of nodes that are not connected by any path represent variables that are conditionally independent of each other. Every node carries a probability function that takes specific values for parent variables as input. The output of this function is the probability distribution of the variable represented by that node. If parent nodes represent Boolean variables, the probability function can be stored as a table with entries for all possible parent combinations. This mathematical framework allows researchers to model complex relationships without needing to store every single joint probability value.
Inference Algorithms And Complexity
Cooper proved exact inference in Bayesian networks is NP-hard while working at Stanford University on large bioinformatic applications in 1990. This result prompted research into approximation algorithms capable of handling tractable approximations to probabilistic inference. Dagum and Luby proved in 1993 that no tractable deterministic algorithm can approximate probabilistic inference within an absolute error less than one half. They also showed that no tractable randomized algorithm can achieve this accuracy with confidence greater than one half. Roth later demonstrated that exact inference is #P-complete and thus as hard as counting satisfying assignments of conjunct normal form formulas. These complexity results suggest that large real-world applications require topological structural constraints or restrictions on conditional probabilities. The bounded variance algorithm developed by Dagum and Luby was the first provable fast approximation algorithm with guarantees on error approximation. It required the minor restriction that conditional probabilities remain bounded away from zero and one by any polynomial of the number of nodes. Variable elimination eliminates non-observed non-query variables one by one through integration or summation. Clique tree propagation caches computation so many variables can be queried simultaneously.