Variational Bayes estimation of hierarchical Dirichlet-multinomial mixtures for text clustering

• Published in: Computational statistics Vol. 38; no. 4; pp. 2015 - 2051
• Main Authors: Bilancia, Massimo, Di Nanni, Michele, Manca, Fabio, Pio, Gianvito
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.12.2023; Springer Nature B.V
• Subjects: Algorithms; Approximation; Clustering; Dirichlet problem; Economic Theory/Quantitative Economics/Mathematical Methods; Inference; Iterative methods; Markov analysis; Markov chains; Mathematics and Statistics; Maxima; Mixtures; Monte Carlo simulation; Original Paper; Probability and Statistics in Computer Science; Probability distribution; Probability Theory and Stochastic Processes; Statistics; Text clustering; Finite mixture models; Variational inference; Dirichlet-multinomial distribution; Bayesian hierarchical modelling
• ISSN: 0943-4062; 1613-9658; 1613-9658
• DOI: 10.1007/s00180-023-01350-8

In this paper, we formulate a hierarchical Bayesian version of the Mixture of Unigrams model for text clustering and approach its posterior inference through variational inference. We compute the explicit expression of the variational objective function for our hierarchical model under a mean-field approximation. We then derive the update equations of a suitable algorithm based on coordinate ascent to find local maxima of the variational target, and estimate the model parameters through the optimized variational hyperparameters. The advantages of variational algorithms over traditional Markov Chain Monte Carlo methods based on iterative posterior sampling are also discussed in detail.