A linearization procedure and a VDM/ECM algorithm for penalized and constrained nonparametric maximum likelihood estimation for mixture models

• Published in: Computational statistics & data analysis Vol. 51; no. 6; pp. 2946 - 2957
• Main Author: Wang, Ji-Ping
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 01.03.2007; Elsevier Science; Elsevier
• Series: Computational Statistics & Data Analysis
• Subjects: Computing algorithm; Constrained NPMLE; Exact sciences and technology; General topics; Mathematics; Mixture models; Multivariate analysis; Nonparametric maximum likelihood; Numerical analysis; Numerical analysis. Scientific computation; Numerical methods in probability and statistics; Parametric inference; Penalized NPMLE; Probability and statistics; Sciences and techniques of general use; Statistics; VDM/ECM; Constrained NPMLE; Nonparametric maximum likelihood; Computing algorithm; VDM/ECM; Mixture models; Penalized NPMLE; Mixed distribution; Data analysis; Linear function; Non parametric method; Non parametric estimation; Constrained estimation; Algorithm; Penalty method; Statistical computation; Distribution function; Maximum likelihood; Linear functional; Likelihood function; Linearization
• ISSN: 0167-9473; 1872-7352
• DOI: 10.1016/j.csda.2006.11.033

Suppose independent observations X i , i = 1 , … , n are observed from a mixture model f ( x ; Q ) ≡ ∫ f ( x ; λ ) d Q ( λ ) , where λ is a scalar and Q ( λ ) is a nondegenerate distribution with an unspecified form. We consider to estimate Q ( λ ) by nonparametric maximum likelihood (NPML) method under two scenarios: (1) the likelihood is penalized by a functional g ( Q ) ; and (2) Q is under a constraint g ( Q ) = g 0 . We propose a simple and reliable algorithm termed VDM/ECM for Q-estimation when the likelihood is penalized by a linear functional. We show this algorithm can be applied to a more general situation where the penalty is not linear, but a function of linear functionals by a linearization procedure. The constrained NPMLE can be found by penalizing the quadratic distance | g ( Q ) - g 0 | 2 under a large penalty factor γ > 0 using this algorithm. The algorithm is illustrated with two real data sets.