Questions about Expectation–maximization algorithm
Short answers, pulled from the story.
Who invented the expectation-maximization algorithm?
Arthur Dempster, Nan Laird, and Donald Rubin named and formalized the EM algorithm in a 1977 paper, but they acknowledged that the method had been proposed many times before in special cases by earlier authors. Prior contributors include Cedric Smith, H.O. Hartley, and Rolf Sundberg, among others.
What is the expectation-maximization algorithm used for?
The EM algorithm is used to find maximum likelihood or maximum a posteriori estimates of parameters in statistical models that involve unobserved latent variables. Applications include medical image reconstruction, hidden Markov models, mixture model fitting, quantitative genetics, psychometrics, and financial risk management.
What are the E step and M step in the EM algorithm?
The E step computes the expected value of the log-likelihood function given current parameter estimates, treating unobserved latent variables probabilistically. The M step then finds the parameter values that maximize that expected log-likelihood, and the two steps alternate until the estimates converge.
Does the EM algorithm always find the global maximum?
No. The EM algorithm is only guaranteed to converge to a local maximum of the likelihood function, not the global maximum. The outcome depends on the starting values, and for multimodal distributions multiple local maxima may exist.
Who proved that the EM algorithm converges?
C.F. Jeff Wu published a correct convergence proof in 1983. The original 1977 Dempster-Laird-Rubin paper had included a flawed convergence argument; Wu's proof extended the guarantee to models outside the exponential family.
What is the alpha-EM algorithm?
The alpha-EM algorithm, developed by Yasuo Matsuyama, is a generalization of the standard EM algorithm that replaces the log-likelihood with an alpha-log likelihood ratio and uses alpha-divergence. It contains the standard log-EM algorithm as a special case and can converge faster by choosing an appropriate alpha value.