Double Happiness: Enhancing the Coupled Gains of L-lag Coupling via Control Variates.

The recently proposed L-lag coupling for unbiased MCMC \citep{biswas2019estimating, jacob2020unbiased} calls for a joint celebration by MCMC practitioners and theoreticians. For practitioners, it circumvents the thorny issue of deciding the burn-in period or when to terminate an MCMC iteration, and opens the door for safe parallel implementation. For theoreticians, it provides a powerful tool to establish elegant and easily estimable bounds on the exact error of MCMC approximation at any finite number of iteration. A serendipitous observation about the bias correcting term led us to introduce naturally available control variates into the L-lag coupling estimators. In turn, this extension enhances the coupled gains of L-lag coupling, because it results in more efficient unbiased estimators as well as a better bound on the total variation error of MCMC iterations, albeit the gains diminish with the numerical value of L. Specifically, the new bound is theoretically guaranteed to never exceed the one given previously. We also argue that L-lag coupling represents a long sought after coupling for the future, breaking a logjam of the coupling-from-the-past type of perfect sampling, by reducing the generally un-achievable requirement of being \textit{perfect} to being \textit{unbiased}, a worthwhile trade-off for ease of implementation in most practical situations. The theoretical analysis is supported by numerical experiments that show tighter bounds and a gain in efficiency when control variates are introduced.