Metropolis-Hastings Algorithm with Delayed Acceptance and Rejection

Source:
By:Yulin Hu

DOI: https://doi.org/10.30564/ret.v2i2.682

Abstract: Metropolis-Hastings algorithms are slowed down by the computation of complex target distributions. To solve this problem, one can use the delayed acceptance Metropolis-Hastings algorithm (MHDA) of Christen and Fox (2005). However, the acceptance rate of a proposed value will always be less than in the standard Metropolis-Hastings. We can fix this problem by using the Metropolis-Hastings algorithm with delayed rejection (MHDR) proposed by Tierney and Mira (1999). In this paper, we combine the ideas of MHDA and MHDR to propose a new MH algorithm, named the Metropolis-Hastings algorithm with delayed acceptance and rejection (MHDAR). The new algorithm reduces the computational cost by division of the prior or likelihood functions and increase the acceptance probability by delay rejection of the second stage. We illustrate those accelerating features by a realistic example. References:

[1] Hastings, W.K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57:97-109. [2] Nicholas Metropolis, Arianna W. Rosenbluth, Marshall N. Rosenbluth, and Augusta H. Teller. (1953). Equation of State Calculations by Fast Computing Machines. J. Chemical Physics, 21:1087-1092. [3] Christen, J. and Fox, C. (2005). Markov chain Monte Carlo using an approximation. Journal of Computational and Graphical Statistics, 14:795-810. [4] Tierney, L. and Mira, A. (1999). Some adaptive Monte Carlo methods for Bayesian inference. Statistics in Medicine, 18:2507-2515. [5] Tierney, L. and Mira, A. (1999). Some adaptive Monte Carlo methods for Bayesian inference. Statistics in Medicine, 18:2507-2515. [6] Banterle, M., Grazian, C., Lee, A., and Robert, C. P. (2015). Accelerating Metropolis-Hastings algorithms by delayed acceptance. URL http://arxiv.org/abs/1406.2660. [7] Robert, C. and Casella, G. (2004). Monte Carlo Statistical Methods. (2nd ed). Springer Verlag, New York. [8] Sherlock, C. and Roberts, G. (2009). Optimal scaling of the random walk Metropolis on elliptically symmetric unimodal targets. Bernoulli, 15:774-798. URL http://dx.doi.org/10.3150/08-BEJ176. [9] Geweke, J. (1992). Evaluating the accuracy of sampling-based approaches to calculating posterior moments, Bayesian Statistics 4 (J. Bernardo, J. Berger, A. Dawid and A. Smith, eds.), Oxford: Oxford University Press, 69-194. [10] Heidelberger P and Welch PD. (1983). Simulation run length control in the presence of an initial transient. Opns Res, 31:1109-44.