Efficient Sequential Monte-Carlo Samplers for Bayesian Inference

• Published in: IEEE transactions on signal processing Vol. 64; no. 5; pp. 1305 - 1319
• Main Authors: Thi Le Thu Nguyen, Septier, Francois, Peters, Gareth W., Delignon, Yves
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.03.2016; The Institute of Electrical and Electronics Engineers, Inc. (IEEE); Institute of Electrical and Electronics Engineers
• Subjects: Algorithms; Applications; Bayes methods; Bayesian inference; complex models; Computation; Computer simulation; Engineering Sciences; Inference; Iterative methods; Kernel; Markov processes; Methodology; Monte Carlo methods; Monte Carlo simulation; Samplers; sequential Monte Carlo sampler; Signal and Image processing; Signal processing; Signal processing algorithms; Statistics; Strategy; Bayesian inference; Sequential Monte Carlo sampler; complex models
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2015.2504342

In many problems, complex non-Gaussian and/or nonlinear models are required to accurately describe a physical system of interest. In such cases, Monte-Carlo algorithms are remarkably flexible and extremely powerful approaches to solve such inference problems. However, in the presence of a high-dimensional and/or multimodal posterior distribution, it is widely documented that standard Monte-Carlo techniques could lead to poor performance. In this paper, the study is focused on a Sequential Monte-Carlo (SMC) sampler framework, a more robust and efficient Monte-Carlo algorithm. Although this approach presents many advantages over traditional Monte-Carlo methods, the potential of this emergent technique is, however, largely underexploited in signal processing. In this paper, we aim at proposing some novel strategies to improve the efficiency and facilitate practical implementation of the SMC sampler. First, we propose an automatic and adaptive strategy that selects the sequence of distributions within the SMC sampler that minimizes the asymptotic variance of the estimator of the posterior normalization constant. The second original contribution we present improves the global efficiency of the SMC sampler by introducing a novel correction mechanism that allows the use of the particles generated through all of the iterations of the algorithm (instead of only particles from the last iteration). This is a significant contribution as it removes the need to discard a large portion of the samples obtained, as is standard in standard SMC methods. This will improve estimation performance in practical settings where the computational budget is important to consider.