The Likelihood Surface of Mixture Models

Abstract

Everyone knows that hidden Markov models (HMM’s) are difficult to estimate because of local maxima in the likelihood surface. ?Yet for many common classes of HMM’s the evidence is that there are no local maxima, at least for long enough samples. ?What do the likelihood surfaces look like for these and other classes of mixture models? These models are ubiquitous in modern applications, yet the most elementary questions about their asymptotic likelihoods remain a mystery. ?I will conjecture that under the right parameterization and with suitable smoothness conditions these models in fact have no local maxima, with probability one, for all sample sizes sufficiently large. Furthermore, I will suggest that the presence of a large number of local maxima is diagnostic of a miss-specified model.