Fast ML Estimation for the Mixture of Factor Analyzers via an ECM Algorithm

• Published in: IEEE transactions on neural networks Vol. 19; no. 11; pp. 1956 - 1961
• Main Authors: Jian-Hua Zhao, Yu, P.L.H.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.11.2008; Institute of Electrical and Electronics Engineers
• Subjects: Algorithm design and analysis; Algorithms; Alternating expectation conditional maximization (AECM); Applied sciences; Artificial Intelligence; Central Processing Unit; Central processing units; Closed-form solution; Computer science; control theory; systems; Computer Simulation; Connectionism. Neural networks; Convergence; Covariance matrix; Electrochemical machining; Exact sciences and technology; expectation conditional maximization (ECM); expectation maximization (EM); Factor Analysis, Statistical; Likelihood Functions; Mathematical analysis; Mathematical models; Maximization; Maximum likelihood estimation; maximum-likelihood estimation (MLE); Missing data; mixture of factor analyzers (MFA); Models, Statistical; Optimization methods; Pattern Recognition, Automated - methods; Stability; Statistics; Vectors (mathematics); Alternating expectation conditional maximization (AECM); maximum-likelihood estimation (MLE); Optimization method; Processing time; Neural network; expectation maximization (EM); Modeling; Exact solution; mixture of factor analyzers (MFA); Missing data; Conditional expectation; expectation conditional maximization (ECM); Maximum likelihood; Incomplete information; EM algorithm; Central unit
• ISSN: 1045-9227; 1941-0093; 1941-0093
• DOI: 10.1109/TNN.2008.2003467

In this brief, we propose a fast expectation conditional maximization (ECM) algorithm for maximum-likelihood (ML) estimation of mixtures of factor analyzers (MFA). Unlike the existing expectation-maximization (EM) algorithms such as the EM in Ghahramani and Hinton, 1996, and the alternating ECM (AECM) in McLachlan and Peel, 2003, where the missing data contains component-indicator vectors as well as latent factors, the missing data in our ECM consists of component-indicator vectors only. The novelty of our algorithm is that closed-form expressions in all conditional maximization (CM) steps are obtained explicitly, instead of resorting to numerical optimization methods. As revealed by experiments, the convergence of our ECM is substantially faster than EM and AECM regardless of whether assessed by central processing unit (CPU) time or number of iterations.