Parameter Estimations and Model Selections in HMMs

Abstract

This series of researches investigate the likelihood-related topics for Hidden Markov models (HMMs). First, inspired by the parallel method in multiple change-point detection, we first propose a marginal-likelihood-based method for determining the number of states for a heterogeneous HMM. The consistency is proven through generalizing the Bernstein von Mises theorem for finite state HMM with further execution on the non-identifiability issue in over-estimated HMM. A computational method for computing the marginal likelihood is provided, along with numerical discussions.
The above approach, however, is relatively restricted to finite state HMM, largely due to the complex nature of HMM likelihood, whose logarithm fails to provide an additive structure like in i.i.d. case, prohibiting many classical approach to be implemented. Therefore, we further propose a representation of the HMM log likelihood and its derivatives, which provides the additive structure allowing most classical methods to be applied under suitable conditions. By such, we are allowed to characterize the Fisher information, Cramer-Rao lower bound and Kullback-Leibler distance for HMMs with general states. We further extend Hajek-Le Cam minimax theorem, AIC, and second-order efficiency of the bootstrap methods to such context. Joint work with Charles Chang, Cheng-Der Fuh, Tianxiao Pang and Chen Yang (in alphabetical order).

Please click here for participating the talk online.