Exploring the generalizability of the optimal 0.234 acceptance rate in random-walk Metropolis and parallel tempering algorithms

Abstract

For random-walk Metropolis (RWM) and parallel tempering (PT) algorithms, an asymptotic acceptance rate of around 0.234 is known to be optimal in the high-dimensional limit. Yet, the practical relevance of this value is uncertain due to the restrictive conditions underlying its derivation. We synthesise previous theoretical advances in extending the 0.234 acceptance rate to more general settings, and demonstrate the applicability and generalizability of the 0.234 theory for practitioners with a comprehensive empirical simulation study on a variety of examples examining how acceptance rates affect Expected Squared Jumping Distance (ESJD). Our experiments show the optimality of the 0.234 acceptance rate for RWM is surprisingly robust even in lower dimensions across various proposal and multimodal target distributions which may or may not have an i.i.d. product density. Experiments on parallel tempering also show that the idealized 0.234 spacing of inverse temperatures may be approximately optimal for low dimensions and non i.i.d. product target densities, and that constructing an inverse temperature ladder with spacings given by a swap acceptance of 0.234 is a viable strategy. However, we observe the applicability of the 0.234 acceptance rate heuristic diminishes for both RWM and PT algorithms below a certain dimension which differs based on the target density, and that inhomogeneously scaled components in the target density further reduces its applicability in lower dimensions.