A Pivotal Allocation-Based Algorithm for Solving the Label-Switching Problem in Bayesian Mixture Models

• Published in: Journal of computational and graphical statistics Vol. 25; no. 1; pp. 266 - 283
• Main Authors: Li, Han, Fan, Xiaodan
• Format: Journal Article
• Language: English
• Published: Alexandria Taylor & Francis 02.01.2016; American Statistical Association, Institute of Mathematical Statistics, and Interface Foundation of North America; Taylor & Francis Ltd
• Subjects: Algorithms; Bayesian analysis; Identifiability; Information theory; Label-switching problem; Markov chain Monte Carlo; Mathematical models; Probability distribution; Studies
• ISSN: 1061-8600; 1537-2715
• DOI: 10.1080/10618600.2014.983643

In Bayesian analysis of mixture models, the label-switching problem occurs as a result of the posterior distribution being invariant to any permutation of cluster indices under symmetric priors. To solve this problem, we propose a novel relabeling algorithm and its variants by investigating an approximate posterior distribution of the latent allocation variables instead of dealing with the component parameters directly. We demonstrate that our relabeling algorithm can be formulated in a rigorous framework based on information theory. Under some circumstances, it is shown to resemble the classical Kullback-Leibler relabeling algorithm and include the recently proposed equivalence classes representatives relabeling algorithm as a special case. Using simulation studies and real data examples, we illustrate the efficiency of our algorithm in dealing with various label-switching phenomena. Supplemental materials for this article are available online.