Genetic-based EM algorithm for learning Gaussian mixture models

• Published in: IEEE transactions on pattern analysis and machine intelligence Vol. 27; no. 8; pp. 1344 - 1348
• Main Authors: Pernkopf, F., Bouchaffra, D.
• Format: Journal Article
• Language: English
• Published: Los Alamitos, CA IEEE 01.08.2005; IEEE Computer Society; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Applied sciences; Artificial Intelligence; Cluster Analysis; clustering; Clustering algorithms; Computer science; control theory; systems; Computer Simulation; Convergence; EM algorithm; Exact sciences and technology; Gaussian; Gaussian mixture models; Genetic algorithm; Genetic algorithms; Handwriting; Image Enhancement - methods; Image Interpretation, Computer-Assisted - methods; Index Terms- Unsupervised learning; Information Storage and Retrieval - methods; Iterative algorithms; Learning; Learning and adaptive systems; Mathematical models; minimum description length; Models, Statistical; Normal Distribution; Numerical Analysis, Computer-Assisted; Optimization; Parametric statistics; Pattern Recognition, Automated - methods; Probability distribution; Robustness; Searching; Space exploration; Statistical learning; Stochastic processes; Studies; Genetic algorithm; MDL criterion; Gaussian mixture models; clustering; minimum description length; EM algorithm; Unsupervised learning
• ISSN: 0162-8828; 1939-3539
• DOI: 10.1109/TPAMI.2005.162

We propose a genetic-based expectation-maximization (GA-EM) algorithm for learning Gaussian mixture models from multivariate data. This algorithm is capable of selecting the number of components of the model using the minimum description length (MDL) criterion. Our approach benefits from the properties of genetic algorithms (GA) and the EM algorithm by combination of both into a single procedure. The population-based stochastic search of the GA explores the search space more thoroughly than the EM method. Therefore, our algorithm enables escaping from local optimal solutions since the algorithm becomes less sensitive to its initialization. The GA-EM algorithm is elitist which maintains the monotonic convergence property of the EM algorithm. The experiments on simulated and real data show that the GA-EM outperforms the EM method since: (1) we have obtained a better MDL score while using exactly the same termination condition for both algorithms; (2) our approach identifies the number of components which were used to generate the underlying data more often than the EM algorithm.