Question

During the execution of the Expectation-Maximization algorithm, what specific function is maximized during the M-step to update the parameter estimates?

Accepted Answer

During the M-step of the Expectation-Maximization algorithm, the function being maximized is the expected value of the log-likelihood function of the complete data, conditioned on the observed data and the current parameter estimates. To understand this, consider that the goal of the algorithm is to find the maximum likelihood estimate of parameters when some data points are hidden or missing. The complete data consists of both the observed data and these hidden variables. Because the hidden variables are unknown, we cannot calculate the log-likelihood directly. Instead, we use the E-step to compute the expected value of the log-likelihood over the possible values of the hidden variables, given our current best guess for the parameters. This creates a surrogate function, often denoted as Q, which serves as a lower bound on the true log-likelihood. In the M-step, which stands for Maximization, we find the new parameter values that maximize this Q function. By increasing this expectation, we mathematically guarantee that the likelihood of the observed data also increases or stays the same. The algorithm repeats these two steps until the parameter estimates converge to a stable state.

Home → All Courses → Science and Mathematics Courses → Computational Statistics → Flashcard

During the execution of the Expectation-Maximization algorithm, what specific function is maximized during the M-step to update the parameter estimates?