Hit and run ARMS: adaptive rejection Metropolis sampling with hit and run random direction

• Published in: Journal of statistical computation and simulation Vol. 86; no. 5; pp. 973 - 985
• Main Authors: Zhang, Huaiye, Wu, Yuefeng, Cheng, Lulu, Kim, Inyoung
• Format: Journal Article
• Language: English
• Published: Abingdon Taylor & Francis 23.03.2016; Taylor & Francis Ltd
• Subjects: adaptive rejection metropolis sampling; Algorithms; Bayesian analysis; empirical Bayesian method; free-knot splines; hit and run algorithm; Multivariate analysis; Probability distribution; Regression; regression splines; Rejection; Samples; Sampling; Sampling techniques; Statistical analysis; Statistical methods; Subspaces
• ISSN: 0094-9655; 1563-5163
• DOI: 10.1080/00949655.2015.1046074

An algorithm for sampling from non-log-concave multivariate distributions is proposed, which improves the adaptive rejection Metropolis sampling (ARMS) algorithm by incorporating the hit and run sampling. It is not rare that the ARMS is trapped away from some subspace with significant probability in the support of the multivariate distribution. While the ARMS updates samples only in the directions that are parallel to dimensions, our proposed method, the hit and run ARMS (HARARMS), updates samples in arbitrary directions determined by the hit and run algorithm, which makes it almost not possible to be trapped in any isolated subspaces. The HARARMS performs the same as ARMS in a single dimension while more reliable in multidimensional spaces. Its performance is illustrated by a Bayesian free-knot spline regression example. We showed that it overcomes the well-known 'lethargy' property and decisively find the global optimal number and locations of the knots of the spline function.