Identifying the number of components in Gaussian mixture models using numerical algebraic geometry

• Published in: Journal of algebra and its applications Vol. 19; no. 11; p. 2050204
• Main Authors: Shirinkam, Sara, Alaeddini, Adel, Gross, Elizabeth
• Format: Journal Article
• Language: English
• Published: Singapore World Scientific Publishing Company 01.11.2020; World Scientific Publishing Co. Pte., Ltd
• Subjects: Algebra; Algorithms; Clustering; Computer simulation; Continuation methods; Criteria; Data smoothing; Identification methods; Iterative methods; Optimization; Polynomials; Probabilistic models; Research Article; Robustness (mathematics); Spline functions; numerical algebraic geometry; smoothing spline; model-based clustering; Mixture models
• ISSN: 0219-4988; 1793-6829; 1793-6829
• DOI: 10.1142/S0219498820502047

Using Gaussian mixture models for clustering is a statistically mature method for clustering in data science with numerous successful applications in science and engineering. The parameters for a Gaussian mixture model (GMM) are typically estimated from training data using the iterative expectation-maximization algorithm, which requires the number of Gaussian components a priori. In this study, we propose two algorithms rooted in numerical algebraic geometry (NAG), namely, an area-based algorithm and a local maxima algorithm, to identify the optimal number of components. The area-based algorithm transforms several GMM with varying number of components into sets of equivalent polynomial regression splines. Next, it uses homotopy continuation methods for evaluating the resulting splines to identify the number of components that is most compatible with the gradient data. The local maxima algorithm forms a set of polynomials by fitting a smoothing spline to a dataset. Next, it uses NAG to solve the system of the first derivatives for finding the local maxima of the resulting smoothing spline, which represent the number of mixture components. The local maxima algorithm also identifies the location of the centers of Gaussian components. Using a real-world case study in automotive manufacturing and extensive simulations, we demonstrate that the performance of the proposed algorithms is comparable with that of Akaike information criterion (AIC) and Bayesian information criterion (BIC), which are popular methods in the literature. We also show the proposed algorithms are more robust than AIC and BIC when the Gaussian assumption is violated.