Bias of Standard Errors in Latent Class Model Applications Using Newton-Raphson and EM Algorithms
Department of Statistics and Operations Research, University of Malta
Msida (MSD 06) Malta
The EM algorithm is a popular method for computing maximum likelihood estimates. It tends to be numerically stable, reduces execution time compared to other estimation procedures and is easy to implement in latent class models. However, the EM algorithm fails to provide a consistent estimator of the standard errors of maximum likelihood estimates in incomplete data applications. Correct standard errors can be obtained by numerical differentiation. The technique requires computation of a complete-data gradient vector and Hessian matrix, but not those associated with the incomplete data likelihood. Obtaining first and second derivatives numerically is computationally very intensive and execution time may become very expensive when fitting Latent class models using a Newton-type algorithm. When the execution time is too high one is motivated to use the EM algorithm solution to initialize the Newton Raphson algorithm. We also investigate the effect on the execution time when a final Newton-Raphson step follows the EM algorithm after convergence. In this paper we compare the standard errors provided by the EM and Newton-Raphson algorithms for two models and analyze how this bias is affected by the number of parameters in the model fit.
-  T. A. Louis, “Finding the Observed Information Matrix when using the EM algorithm,” Journal of the Royal Statistical Society, 44, pp. 226-233, 1982.
-  R. A. Fisher, “Theory of Statistical Estimation,” Proc. Camb. Phil. Society, 22, pp. 700-725, 1925.
-  I. Meilijson, “A Fast Improvement to the EM algorithm on its Own Term,” Journal of the Royal Statistical Society, 51, pp. 127-138, 1989.
-  X. L. Meng and D. B. Rubin, “Using EM to obtain asymptotic variance-covariance matrices: the SEM algorithm,” Journal of American Statistical Association, 86, pp. 899-909, 1991.
-  A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum Likelihood from Incomplete Data via the EM algorithm,” Journal of the Royal Statistical Society, B, 39, pp. 1-38, 1977.
-  M. R. Segal, P. Bacchetti, and N. P. Jewell, “Variance for Maximum Penalized Likelihood estimates obtained via the EM algorithm,” Journal of the Royal Statistical Society, B, 56, pp. 345-352, 1994.
-  C. E. McCulloch, “Maximum Likelihood Variance components estimation for Binary Data,” Journal of the American Statistical Association, 89, pp. 330-335, 1998.
-  M. Jamshidian and R. I. Jennrich, “Standard Errors for EM Estimation,” Journal of the Royal Statistical Society, B, pp. 257-270, 2000.
-  A. Agresti, “Categorical Data Analysis,” A Wiley-Interscience Publication, 2002.
-  P. McCullagh, “Regression Models for Ordinal Data,” J.R. Statistical. Soc, B, 42, pp. 109-142, 1980.