Acceleration of the EM algorithm: P-EM versus epsilon algorithm

• Published in: Computational statistics & data analysis Vol. 56; no. 12; pp. 4122 - 4137
• Main Authors: Berlinet, A.F., Roland, Ch
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.12.2012
• Subjects: Acceleration; Algorithms; Computation; Convergence; Data processing; EM algorithm; Epsilon algorithm; Fixed point iteration; Mathematical analysis; Mathematical models; Statistics; Vectors (mathematics); Epsilon algorithm; Acceleration; EM algorithm; Fixed point iteration; Convergence
• ISSN: 0167-9473; 1872-7352
• DOI: 10.1016/j.csda.2012.03.005

Among recent methods designed for accelerating the EM algorithm without any modification in the structure of EM or in the statistical model, the parabolic acceleration (P-EM) has proved its efficiency. It does not involve any computation of gradient or hessian matrix and can be used as an additional software component of any fixed point algorithm maximizing some objective function. The vector epsilon algorithm was introduced to reach the same goals. Through geometric considerations, the relationships between the outputs of an improved version of P-EM and those of the vector epsilon algorithm are established. This sheds some light on their different behaviours and explains why the parabolic acceleration of EM outperforms its competitor in most numerical experiments. A detailed analysis of its trajectories in a variety of real or simulated data shows the ability of P-EM to choose the most efficient paths to the global maximum of the likelihood.