scholarly journals Model selection on solid ground: Rigorous comparison of nine ways to evaluate Bayesian model evidence

2014 ◽  
Vol 50 (12) ◽  
pp. 9484-9513 ◽  
Author(s):  
Anneli Schöniger ◽  
Thomas Wöhling ◽  
Luis Samaniego ◽  
Wolfgang Nowak
Keyword(s):  
Model Selection ◽  
Bayesian Model ◽  
Solid Ground ◽  
Model Evidence

scholarly journals Thermodynamic integration for dynamic causal models

2018 ◽  
Author(s):  
Eduardo A. Aponte ◽  
Sudhir Raman ◽  
Stefan Frässle ◽  
Jakob Heinzle ◽  
Will D. Penny ◽  
...  
Keyword(s):  
Model Selection ◽  
Bayesian Model ◽  
Model Comparison ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Computation Time ◽  
Thermodynamic Integration ◽  
Causal Modeling ◽  
Single Subject ◽  
Model Evidence

AbstractIn generative modeling of neuroimaging data, such as dynamic causal modeling (DCM), one typically considers several alternative models, either to determine the most plausible explanation for observed data (Bayesian model selection) or to account for model uncertainty (Bayesian model averaging). Both procedures rest on estimates of the model evidence, a principled trade-off between model accuracy and complexity. In DCM, the log evidence is usually approximated using variational Bayes (VB) under the Laplace approximation (VBL). Although this approach is highly efficient, it makes distributional assumptions and can be vulnerable to local extrema. An alternative to VBL is Markov Chain Monte Carlo (MCMC) sampling, which is asymptotically exact but orders of magnitude slower than VB. This has so far prevented its routine use for DCM.This paper makes four contributions. First, we introduce a powerful MCMC scheme – thermodynamic integration (TI) – to neuroimaging and present a derivation that establishes a theoretical link to VB. Second, this derivation is based on a tutorial-like introduction to concepts of free energy in physics and statistics. Third, we present an implementation of TI for DCM that rests on population MCMC. Fourth, using simulations and empirical functional magnetic resonance imaging (fMRI) data, we compare log evidence estimates obtained by TI, VBL, and other MCMC-based estimators (prior arithmetic mean and posterior harmonic mean). We find that model comparison based on VBL gives reliable results in most cases, justifying its use in standard DCM for fMRI. Furthermore, we demonstrate that for complex and/or nonlinear models, TI may provide more robust estimates of the log evidence. Importantly, accurate estimates of the model evidence can be obtained with TI in acceptable computation time. This paves the way for using DCM in scenarios where the robustness of single-subject inference and model selection becomes paramount, such as differential diagnosis in clinical applications.

scholarly journals Making Steppingstones out of Stumbling Blocks: A Bayesian Model Evidence Estimator with Application to Groundwater Transport Model Selection

Water ◽  
2019 ◽  
Vol 11 (8) ◽  
pp. 1579 ◽  
Author(s):  
Elshall ◽  
Ye
Keyword(s):  
Model Selection ◽  
Bayesian Model ◽  
Goodness Of Fit ◽  
Transport Model ◽  
Model Complexity ◽  
Accurate Estimation ◽  
Observation Data ◽  
Sampling Errors ◽  
Groundwater Transport ◽  
Model Evidence

Bayesian model evidence (BME) is a measure of the average fit of a model to observation data given all the parameter values that the model can assume. By accounting for the trade-off between goodness-of-fit and model complexity, BME is used for model selection and model averaging purposes. For strict Bayesian computation, the theoretically unbiased Monte Carlo based numerical estimators are preferred over semi-analytical solutions. This study examines five BME numerical estimators and asks how accurate estimation of the BME is important for penalizing model complexity. The limiting cases for numerical BME estimators are the prior sampling arithmetic mean estimator (AM) and the posterior sampling harmonic mean (HM) estimator, which are straightforward to implement, yet they result in underestimation and overestimation, respectively. We also consider the path sampling methods of thermodynamic integration (TI) and steppingstone sampling (SS) that sample multiple intermediate distributions that link the prior and the posterior. Although TI and SS are theoretically unbiased estimators, they could have a bias in practice arising from numerical implementation. For example, sampling errors of some intermediate distributions can introduce bias. We propose a variant of SS, namely the multiple one-steppingstone sampling (MOSS) that is less sensitive to sampling errors. We evaluate these five estimators using a groundwater transport model selection problem. SS and MOSS give the least biased BME estimation at an efficient computational cost. If the estimated BME has a bias that covariates with the true BME, this would not be a problem because we are interested in BME ratios and not their absolute values. On the contrary, the results show that BME estimation bias can be a function of model complexity. Thus, biased BME estimation results in inaccurate penalization of more complex models, which changes the model ranking. This was less observed with SS and MOSS as with the three other methods.

scholarly journals Bayesian model selection on scalar ε -field dark energy

2021 ◽  
Vol 103 (4) ◽  
Author(s):  
J. Alberto Vázquez ◽  
David Tamayo ◽  
Anjan A. Sen ◽  
Israel Quiros
Keyword(s):  
Dark Energy ◽  
Model Selection ◽  
Bayesian Model ◽  
Bayesian Model Selection

scholarly journals Accounting for cell lineage and sex effects in the identification of cell-specific DNA methylation using a Bayesian model selection algorithm

PLoS ONE ◽  
2017 ◽  
Vol 12 (9) ◽  
pp. e0182455 ◽  
Author(s):  
Nicole White ◽  
Miles Benton ◽  
Daniel Kennedy ◽  
Andrew Fox ◽  
Lyn Griffiths ◽  
...  
Keyword(s):  
Dna Methylation ◽  
Model Selection ◽  
Bayesian Model ◽  
Cell Lineage ◽  
Bayesian Model Selection ◽  
Selection Algorithm ◽  
Sex Effects
Sign in / Sign up

Export Citation Format

Share Document