Preconditioned Crank‐Nicolson Markov Chain Monte Carlo Coupled With Parallel Tempering: An Efficient Method for Bayesian Inversion of Multi‐Gaussian Log‐Hydraulic Conductivity Fields

2020 ◽  
Vol 56 (8) ◽  
Author(s):  
Teng Xu ◽  
Sebastian Reuschen ◽  
Wolfgang Nowak ◽  
Harrie‐Jan Hendricks Franssen
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Hydraulic Conductivity ◽  
Efficient Method ◽  
Parallel Tempering ◽  
Bayesian Inversion ◽  
Crank Nicolson

A Distributed Computer System for Parallel Markov Chain Monte Carlo (MCMC)

2018 ◽  
Author(s):  
Michael Hynes
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Parameter Space ◽  
Heterogeneous Computing ◽  
Peer To Peer ◽  
Parallel Tempering ◽  
Single Chain ◽  
Distributed Computing Systems ◽  
Mcmc Algorithms

A ubiquitous problem in physics is to determine expectation values of observables associated with a system. This problem is typically formulated as an integration of some likelihood over a multidimensional parameter space. In Bayesian analysis, numerical Markov Chain Monte Carlo (MCMC) algorithms are employed to solve such integrals using a fixed number of samples in the Markov Chain. In general, MCMC algorithms are computationally expensive for large datasets and have difficulties sampling from multimodal parameter spaces. An MCMC implementation that is robust and inexpensive for researchers is desired. Distributed computing systems have shown the potential to act as virtual supercomputers, such as in the SETI@home project in which millions of private computers participate. We propose that a clustered peer-to-peer (P2P) computer network serves as an ideal structure to run Markovian state exchange algorithms such as Parallel Tempering (PT). PT overcomes the difficulty in sampling from multimodal distributions by running multiple chains in parallel with different target distributions andexchanging their states in a Markovian manner. To demonstrate the feasibility of peer-to-peer Parallel Tempering (P2P PT), a simple two-dimensional dataset consisting of two Gaussian signals separated by a region of low probability was used in a Bayesian parameter fitting algorithm. A small connected peer-to-peer network was constructed using separate processes on a linux kernel, and P2P PT was applied to the dataset. These sampling results were compared with those obtained from sampling the parameter space with a single chain. It was found that the single chain was unable to sample both modes effectively, while the P2P PT method explored the target distribution well, visiting both modes approximately equally. Future work will involve scaling to many dimensions and large networks, and convergence conditions with highly heterogeneous computing capabilities of members within the network.

scholarly journals Accelerating parallel tempering: Quantile tempering algorithm (QuanTA)

2019 ◽  
Vol 51 (03) ◽  
pp. 802-834 ◽  
Author(s):  
Nicholas G. Tawn ◽  
Gareth O. Roberts
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Theoretical Analysis ◽  
Parallel Tempering ◽  
Mcmc Methods ◽  
Regularity Conditions ◽  
Population Approach ◽  
Multimodal Problems ◽  
Established Population

AbstractIt is well known that traditional Markov chain Monte Carlo (MCMC) methods can fail to effectively explore the state space for multimodal problems. Parallel tempering is a well-established population approach for such target distributions involving a collection of particles indexed by temperature. However, this method can suffer dramatically from the curse of dimensionality. In this paper we introduce an improvement on parallel tempering called QuanTA. A comprehensive theoretical analysis quantifying the improved efficiency and scalability of the approach is given. Under weak regularity conditions, QuanTA gives accelerated mixing through the temperature space. Empirical evidence of the effectiveness of this new algorithm is illustrated on canonical examples.

Sign in / Sign up

Export Citation Format

Share Document