A Two-Layer Mixture Model of Gaussian Process Functional Regressions and Its MCMC EM Algorithm

• Published in: IEEE transaction on neural networks and learning systems Vol. 29; no. 10; pp. 4894 - 4904
• Main Authors: Wu, Di, Ma, Jinwen
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.10.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Approximation algorithms; Clustering; Clustering algorithms; Computer simulation; Curve clustering and prediction; Data models; EM algorithm; Estimating techniques; Gaussian process; Gaussian process (GP); Machine learning; Markov chains; Mixture models; mixture of Gaussian processes (MGP/mix-GP); Monte Carlo methods; Monte Carlo simulation; Parameter estimation; parameter learning; Prediction algorithms; Probabilistic models; Stochastic models; Stochastic processes; Stochasticity
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2017.2782711

The mixture of Gaussian processes (GPs) is capable of learning any general stochastic process based on a given set of (sample) curves for the regression and prediction problems. However, it is ineffective for curve clustering and prediction, when the sample curves are derived from different stochastic processes as independent sources linearly mixed together. In this paper, we propose a two-layer mixture model of GP functional regressions (GPFRs) to describe such a mixture of general stochastic processes or independent sources, especially for curve clustering and prediction. Specifically, in the lower layer, the mixture of GPFRs (MGPFRs) is developed for a cluster (or class) of curves within the input space . In the higher layer, the mixture of MGPFRs is further established to divide the curves into clusters according to its components in the output space . For the parameter estimation of the two-layer mixture of GPFRs, we develop a Monte Carlo EM algorithm based on a Monte Carlo Markov chain (MCMC) method, in short, the MCMC EM algorithm. We validate the hierarchical mixture of GPFRs and MCMC EM algorithm using synthetic and real-world data sets. Our results show that our new model outperforms the conventional mixture models in curve clustering and prediction.