Variational Bayesian Methods for a Tree-Structured Stick-Breaking Process Mixture of Gaussians by Application of the Bayes Codes for Context Tree Models

• Published in: International Symposium on Information Theory and its Applications (Online) pp. 75 - 80
• Main Author: Nakahara, Yuta
• Format: Conference Proceeding
• Language: English
• Published: The Institute of Electronics, Information and Communication Engineers 10.11.2024
• Subjects: Algorithms; Bayes methods; Benchmark testing; Computational efficiency; Computational modeling; Context modeling; Encoding; Mathematical models; Mixture models; Monte Carlo methods
• ISSN: 2689-5854
• DOI: 10.23919/ISITA60732.2024.10858231

The tree-structured stick-breaking process (TS-SBP) mixture model is a non-parametric Bayesian model that can represent tree-like hierarchical structures among the mixture components. For TS-SBP mixture models, only a Markov chain Monte Carlo (MCMC) method has been proposed and any variational Bayesian (VB) methods has not been proposed. In general, MCMC methods are computationally more expensive than VB methods. Therefore, we require a large computational cost to learn the TS-SBP mixture model. In this paper, we propose a learning algorithm with less computational cost for the TS-SBP mixture of Gaussians by using the VB method under an assumption of finite tree width and depth. When constructing such VB method, the main challenge is efficient calculation of a sum over all possible trees. To solve this challenge, we utilizes a subroutine in the Bayes coding algorithm for context tree models. We confirm the computational efficiency of our VB method through an experiments on a benchmark dataset.