Robust Gaussian process modeling using EM algorithm

• Published in: Journal of process control Vol. 42; pp. 125 - 136
• Main Authors: Ranjan, Rishik, Huang, Biao, Fatehi, Alireza
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.06.2016
• Subjects: EM algorithm; Gaussian process regression; Outliers; Robust Gaussian process regression; Steam Assisted Gravity Drainage (SAGD); Outliers; Robust Gaussian process regression; Gaussian process regression; EM algorithm; Steam Assisted Gravity Drainage (SAGD)
• ISSN: 0959-1524; 1873-2771
• DOI: 10.1016/j.jprocont.2016.04.003

•Propose an EM algorithm method for robust Gaussian process model identification.•Two noise distributions have been considered: Student's t and Laplace.•Propose approach is numerically stable and guaranteed to converge.•Dynamic model of a water treatment unit was constructed using proposed method. Gaussian process (GP) regression is a fully probabilistic method for performing non-linear regression. In a Bayesian framework, regression models can be made robust by using heavy-tailed distributions instead of using normal distribution for modeling noise. This work focuses on estimation of parameters for robust GP regression. In literature, these are learned by maximizing the approximate marginal likelihood of data. However, gradient-based optimization algorithms which are used for this purpose can be unstable or may require tuning. In this work, an EM algorithm based approach is derived and implemented to infer the parameters. The pros and cons of the two approaches are analyzed. The advantage of EM algorithm lies in its ease of implementation and theoretical guarantees of numerical stability and convergence while its prediction performance is still comparable to gradient-based approaches. In some cases EM algorithm may be slow to converge. To circumvent this issue a faster EM based approach known as Expectation Conjugate Gradient (ECG) is implemented on robust GP regression. Finally, the proposed EM approach to robust GP regression is validated using an industrial data set.