scholarly journals Joining forces of Bayesian and frequentist methodology: a study for inference in the presence of non-identifiability

2013 ◽  
Vol 371 (1984) ◽  
pp. 20110544 ◽  
Author(s):  
Andreas Raue ◽  
Clemens Kreutz ◽  
Fabian Joachim Theis ◽  
Jens Timmer
Keyword(s):  
Experimental Data ◽  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Probability Distribution ◽  
Cell Biology ◽  
Posterior Probability ◽  
Monte Carlo Sampling ◽  
Posterior Probability Distribution ◽  
Model Predictions

Increasingly complex applications involve large datasets in combination with nonlinear and high-dimensional mathematical models. In this context, statistical inference is a challenging issue that calls for pragmatic approaches that take advantage of both Bayesian and frequentist methods. The elegance of Bayesian methodology is founded in the propagation of information content provided by experimental data and prior assumptions to the posterior probability distribution of model predictions. However, for complex applications, experimental data and prior assumptions potentially constrain the posterior probability distribution insufficiently. In these situations, Bayesian Markov chain Monte Carlo sampling can be infeasible. From a frequentist point of view, insufficient experimental data and prior assumptions can be interpreted as non-identifiability. The profile-likelihood approach offers to detect and to resolve non-identifiability by experimental design iteratively. Therefore, it allows one to better constrain the posterior probability distribution until Markov chain Monte Carlo sampling can be used securely. Using an application from cell biology, we compare both methods and show that a successive application of the two methods facilitates a realistic assessment of uncertainty in model predictions.

scholarly journals Seismic Tomography Using Variational Inference Methods

2020 ◽  
Author(s):  
Xin Zhang ◽  
Andrew Curtis
Keyword(s):  
Monte Carlo ◽  
Probability Distribution ◽  
Seismic Tomography ◽  
Posterior Probability ◽  
Sampling Methods ◽  
Probability Distributions ◽  
Monte Carlo Sampling ◽  
Variational Inference ◽  
Posterior Probability Distribution ◽  
Inference Methods

<p><span>In a variety of geoscientific applications we require maps of subsurface properties together with the corresponding maps of uncertainties to assess their reliability. Seismic tomography is a method that is widely used to generate those maps. Since tomography is significantly nonlinear, Monte Carlo sampling methods are often used for this purpose, but they are generally computationally intractable for large data sets and high-dimensionality parameter spaces. To extend uncertainty analysis to larger systems, we introduce variational inference methods to conduct seismic tomography. In contrast to Monte Carlo sampling, variational methods solve the Bayesian inference problem as an optimization problem yet still provide fully nonlinear, probabilistic results. This is achieved by minimizing the Kullback-Leibler (KL) divergence between approximate and target probability distributions within a predefined family of probability distributions.</span></p><p><span>We introduce two variational inference methods: automatic differential variational inference (ADVI) and Stein variational gradient descent (SVGD). In ADVI a Gaussian probability distribution is assumed and optimized to approximate the posterior probability distribution. In SVGD a smooth transform is iteratively applied to an initial probability distribution to obtain an approximation to the posterior probability distribution. At each iteration the transform is determined by seeking the steepest descent direction that minimizes the KL-divergence. </span></p><p><span>We apply the two variational inference methods to 2D travel time tomography using both synthetic and real data, and compare the results to those obtained from two different Monte Carlo sampling methods: Metropolis-Hastings Markov chain Monte Carlo (MH-McMC) and reversible jump Markov chain Monte Carlo (rj-McMC). The results show that ADVI provides a biased approximation because of its Gaussian approximation, whereas SVGD produces more accurate approximations to the results of MH-McMC. In comparison rj-McMC produces smoother mean velocity models and lower standard deviations because the parameterization used in rj-McMC (Voronoi cells) imposes prior restrictions on the pixelated form of models: all pixels within each Voronoi cell have identical velocities. This suggests that the results of rj-McMC need to be interpreted in the light of the specific prior information imposed by the parameterization. Both variational methods estimate the posterior distribution at significantly lower computational cost, provided that gradients of parameters with respect to data can be calculated efficiently. We therefore expect that the methods can be applied fruitfully to many other types of geophysical inverse problems.</span></p>

scholarly journals Bayesian Multiple Emitter Fitting using Reversible Jump Markov Chain Monte Carlo

2019 ◽  
Author(s):  
Mohamadreza Fazel ◽  
Michael J. Wester ◽  
Hanieh Mazloom-Farsibaf ◽  
Marjolein B. M. Meddens ◽  
Alexandra Eklund ◽  
...  
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Single Molecule ◽  
Posterior Probability ◽  
Super Resolution ◽  
Image Data ◽  
Reversible Jump ◽  
Posterior Probability Distribution ◽  
Resolution Imaging

In single molecule localization-based super-resolution imaging, high labeling density or the desire for greater data collection speed can lead to clusters of overlapping emitter images in the raw super-resolution image data. We describe a Bayesian inference approach to multiple-emitter fitting that uses Reversible Jump Markov Chain Monte Carlo to identify and localize the emitters in dense regions of data. This formalism can take advantage of any prior information, such as emitter intensity and density. The output is both a posterior probability distribution of emitter locations that includes uncertainty in the number of emitters and the background structure, and a set of coordinates and uncertainties from the most probable model.

scholarly journals Nonlinear Parameter Estimation: Comparison of an Ensemble Kalman Smoother with a Markov Chain Monte Carlo Algorithm

2012 ◽  
Vol 140 (6) ◽  
pp. 1957-1974 ◽  
Author(s):  
Derek J. Posselt ◽  
Craig H. Bishop
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Parameter Uncertainty ◽  
Posterior Probability ◽  
Monte Carlo Algorithm ◽  
Parameter Estimates ◽  
Posterior Probability Distribution ◽  
Kalman Smoother ◽  
Non Gaussian

Abstract This paper explores the temporal evolution of cloud microphysical parameter uncertainty using an idealized 1D model of deep convection. Model parameter uncertainty is quantified using a Markov chain Monte Carlo (MCMC) algorithm. A new form of the ensemble transform Kalman smoother (ETKS) appropriate for the case where the number of ensemble members exceeds the number of observations is then used to obtain estimates of model uncertainty associated with variability in model physics parameters. Robustness of the parameter estimates and ensemble parameter distributions derived from ETKS is assessed via comparison with MCMC. Nonlinearity in the relationship between parameters and model output gives rise to a non-Gaussian posterior probability distribution for the parameters that exhibits skewness early and multimodality late in the simulation. The transition from unimodal to multimodal posterior probability density function (PDF) reflects the transition from convective to stratiform rainfall. ETKS-based estimates of the posterior mean are shown to be robust, as long as the posterior PDF has a single mode. Once multimodality manifests in the solution, the MCMC posterior parameter means and variances differ markedly from those from the ETKS. However, it is also shown that if the ETKS is given a multimode prior ensemble, multimodality is preserved in the ETKS posterior analysis. These results suggest that the primary limitation of the ETKS is not the inability to deal with multimodal, non-Gaussian priors. Rather it is the inability of the ETKS to represent posterior perturbations as nonlinear functions of prior perturbations that causes the most profound difference between MCMC posterior PDFs and ETKS posterior PDFs.

scholarly journals Bayesian Multiple Emitter Fitting using Reversible Jump Markov Chain Monte Carlo

2019 ◽  
Vol 9 (1) ◽  
Author(s):  
Mohamadreza Fazel ◽  
Michael J. Wester ◽  
Hanieh Mazloom-Farsibaf ◽  
Marjolein B. M. Meddens ◽  
Alexandra S. Eklund ◽  
...  
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Single Molecule ◽  
Posterior Probability ◽  
Super Resolution ◽  
Image Data ◽  
Reversible Jump ◽  
Posterior Probability Distribution ◽  
Resolution Imaging

Abstract In single molecule localization-based super-resolution imaging, high labeling density or the desire for greater data collection speed can lead to clusters of overlapping emitter images in the raw super-resolution image data. We describe a Bayesian inference approach to multiple-emitter fitting that uses Reversible Jump Markov Chain Monte Carlo to identify and localize the emitters in dense regions of data. This formalism can take advantage of any prior information, such as emitter intensity and density. The output is both a posterior probability distribution of emitter locations that includes uncertainty in the number of emitters and the background structure, and a set of coordinates and uncertainties from the most probable model.

scholarly journals A Monte Carlo approach for determining cluster evaporation rates from concentration measurements

2016 ◽  
Author(s):  
Oona Kupiainen-Määttä
Keyword(s):  
Experimental Data ◽  
Monte Carlo ◽  
Markov Chain ◽  
Sulfuric Acid ◽  
Markov Chain Monte Carlo ◽  
Mass Spectrometer ◽  
Large Fraction ◽  
Cluster Formation ◽  
Unknown Parameters ◽  
Ammonia Clusters

Abstract. Evaporation rates of small negatively charged sulfuric acid–ammonia clusters are determined by combining detailed cluster formation simulations with cluster distributions measured at CLOUD. The analysis is performed by varying the evaporation rates with Markov chain Monte Carlo (MCMC), running cluster formation simulations with each new set of evaporation rates and comparing the obtained cluster distributions to the measurements. In a second set of simulations, the fragmentation of clusters in the mass spectrometer due to energetic collisions is studied by treating also the fragmentation probabilities as unknown parameters and varying them with MCMC. This second set of simulations results in a better fit to the experimental data, suggesting that a large fraction of the observed HSO4− and HSO4− ⋅ H2SO4 signals may result from fragmentation of larger clusters, most importantly the HSO4− ⋅ (H2SO4)2 trimer.

scholarly journals Automated Parameter Blocking for Efficient Markov Chain Monte Carlo Sampling

2017 ◽  
Vol 12 (2) ◽  
pp. 465-490 ◽  
Author(s):  
Daniel Turek ◽  
Perry de Valpine ◽  
Christopher J. Paciorek ◽  
Clifford Anderson-Bergman
Keyword(s):  
Monte Carlo ◽  
Markov Chain ◽  
Markov Chain Monte Carlo ◽  
Monte Carlo Sampling
Sign in / Sign up

Export Citation Format

Share Document