The Multiple-Try Method and Local Optimization in Metropolis Sampling

• Published in: Journal of the American Statistical Association Vol. 95; no. 449; pp. 121 - 134
• Main Authors: Liu, Jun S., Liang, Faming, Wong, Wing Hung
• Format: Journal Article
• Language: English
• Published: Alexandria, VA Taylor & Francis Group 01.03.2000; American Statistical Association; Taylor & Francis Ltd
• Subjects: Adaptive direction sampling; Algorithms; Autocorrelation; Conjugate gradient; Damped sinusoidal; Datasets; Determinism; Distribution theory; Exact sciences and technology; Gibbs sampling; Griddy Gibbs sampler; Histograms; Hit-and-run algorithm; Linear inference, regression; Markov analysis; Markov chain Monte Carlo; Markov chains; Markov processes; Markovian processes; Mathematical economics; Mathematics; Metropolis; Metropolis algorithm; Metropolitan areas; Mixture model; Monte Carlo simulation; Optimization; Orientational bias Monte Carlo; Probability and statistics; Probability theory and stochastic processes; Random sampling; Sampling; Sampling distributions; Sampling theory, sample surveys; Sciences and techniques of general use; Statistics; Tanneries; Theory and Methods; Time series; Transition rules; Markov chain; Conjugate gradient method; Monte Carlo method; Adaptive directional sampling algorithm; Gridy Gibbs sampler; Adaptive estimation; Gibbs sampling; Sampling; Metropolis Hastings sampler; Markov chain Monte Carlo simulation; Mixture; Sinusoid
• ISSN: 0162-1459; 1537-274X
• DOI: 10.1080/01621459.2000.10473908

This article describes a new Metropolis-like transition rule, the multiple-try Metropolis, for Markov chain Monte Carlo (MCMC) simulations. By using this transition rule together with adaptive direction sampling, we propose a novel method for incorporating local optimization steps into a MCMC sampler in continuous state-space. Numerical studies show that the new method performs significantly better than the traditional Metropolis-Hastings (M-H) sampler. With minor tailoring in using the rule, the multiple-try method can also be exploited to achieve the effect of a griddy Gibbs sampler without having to bear with griddy approximations, and the effect of a hit-and-run algorithm without having to figure out the required conditional distribution in a random direction.