A Stationary Stochastic Approximation Algorithm for Estimation in Generalized Linear Mixed Models

Abstract

Estimation in generalized linear models is challenging because the likelihood is an integral without closed form. Among those leading solutions the likelihood is approximated and the maximum likelihood estimate (MLE) can only be reached with error. The simultaneous perturbation stochastic approximation (SPSA) algorithm is designed to find the exact MLE under the same circumstances but provides no error bound if the algorithm is stopped in a finite steps. In order to estimate MLE properly with an error bound, we design the stationary SPSA (SSPSA) algorithm. Assuming that the marginal likelihood is quadratic around the MLE, the SSPSA takes the form of a random coefficient vector autoregressive model. Under some mild conditions, the algorithm yields a stationary sequence where the mean of this sequence is asymptotically unbiased and consistent to the MLE.