Reversible Markov Chains

This chapter is dedicated to the Gaussian approximation of a reversible Markov chain. Regarding this problem, the coefficients of dependence for reversible Markov chains are actually the covariances between the variables. We present here the traditional form of the martingale approximation including forward and backward martingale approximations. Special attention is given to maximal inequalities which are building blocks for the functional limit theorems. When the covariances are summable we present the functional central limit theorem under the standard normalization √n. When the variance of the partial sums are regularly varying with n, we present the functional CLT using as normalization the standard deviation of partial sums. Applications are given to the Metropolis–Hastings algorithm.

Assessing significance in a Markov chain without mixing

We present a statistical test to detect that a presented state of a reversible Markov chain was not chosen from a stationary distribution. In particular, given a value function for the states of the Markov chain, we would like to show rigorously that the presented state is an outlier with respect to the values, by establishing a p value under the null hypothesis that it was chosen from a stationary distribution of the chain. A simple heuristic used in practice is to sample ranks of states from long random trajectories on the Markov chain and compare these with the rank of the presented state; if the presented state is a 0.1% outlier compared with the sampled ranks (its rank is in the bottom 0.1% of sampled ranks), then this observation should correspond to a p value of 0.001. This significance is not rigorous, however, without good bounds on the mixing time of the Markov chain. Our test is the following: Given the presented state in the Markov chain, take a random walk from the presented state for any number of steps. We prove that observing that the presented state is an ε-outlier on the walk is significant at p=2ε under the null hypothesis that the state was chosen from a stationary distribution. We assume nothing about the Markov chain beyond reversibility and show that significance at p≈ε is best possible in general. We illustrate the use of our test with a potential application to the rigorous detection of gerrymandering in Congressional districting.