Bayesian Model Averaging for Accounting for Model Selection Uncertainty with an Application for Predicting Net Radiation

2009 ◽  
Author(s):  
Christopher H Hay ◽  
Suat Irmak
Keyword(s):  
Model Selection ◽  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Net Radiation ◽  
Model Selection Uncertainty

Models and model selection uncertainty in estimating growth rates of endangered freshwater mussel populations

2008 ◽  
Vol 65 (11) ◽  
pp. 2389-2398 ◽  
Author(s):  
Yan Jiao ◽  
Richard Neves ◽  
Jess Jones
Keyword(s):  
Endangered Species ◽  
Model Selection ◽  
Population Growth ◽  
Growth Rates ◽  
Bayesian Model ◽  
Goodness Of Fit ◽  
Bayesian Model Averaging ◽  
Deviance Information Criterion ◽  
Model Averaging ◽  
Model Selection Uncertainty

Appropriate inference of population status for endangered species is extremely important. Using a single model for estimating population growth rates is typically inadequate for assessing endangered species because inferences based on only one “best” model ignore model uncertainty. In this study, the endangered dromedary pearlymussel ( Dromus dromas ) in the Clinch and Powell rivers of eastern Tennessee, USA, was used as an example to demonstrate the importance of multiple models, with consideration of environmental noises for evaluating population growth. Our results showed that more than one model deserves consideration in making inferences of population growth rate. A Bayesian model averaging approach was used to make inferences by weighting each model using the deviance information criterion. To test the uncertainty resulting from model selection and the efficiency of the Bayesian averaging approach, a simulation study was conducted on the dromedary pearlymussel populations, which showed that model selection uncertainty is very high. The results of these tests lead us to recommend using Bayesian model averaging to assess population growth status for endangered species, by balancing goodness-of-fit and selection uncertainty among alternate models.

scholarly journals Bayesian Model Averaging to Account for Model Uncertainty in Estimates of a Vaccine's Effectiveness

2021 ◽  
Author(s):  
Carlos R Oliveira ◽  
Eugene D Shapiro ◽  
Daniel M Weinberger
Keyword(s):  
Model Selection ◽  
Model Uncertainty ◽  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Selection Methods ◽  
Final Model ◽  
Negative Case ◽  
Confounder Selection ◽  
Control Study

Vaccine effectiveness (VE) studies are often conducted after the introduction of new vaccines to ensure they provide protection in real-world settings. Although susceptible to confounding, the test-negative case-control study design is the most efficient method to assess VE post-licensure. Control of confounding is often needed during the analyses, which is most efficiently done through multivariable modeling. When a large number of potential confounders are being considered, it can be challenging to know which variables need to be included in the final model. This paper highlights the importance of considering model uncertainty by re-analyzing a Lyme VE study using several confounder selection methods. We propose an intuitive Bayesian Model Averaging (BMA) framework for this task and compare the performance of BMA to that of traditional single-best-model-selection methods. We demonstrate how BMA can be advantageous in situations when there is uncertainty about model selection by systematically considering alternative models and increasing transparency.

scholarly journals How to improve parameter estimates in GLM-based fMRI data analysis: cross-validated Bayesian model averaging

2016 ◽  
Author(s):  
Joram Soch ◽  
Achim Pascal Meyer ◽  
John-Dylan Haynes ◽  
Carsten Allefeld
Keyword(s):  
Data Analysis ◽  
Model Selection ◽  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Bayesian Model Selection ◽  
Fmri Data ◽  
Parameter Estimates ◽  
Fmri Data Analysis ◽  
Level Analysis

AbstractIn functional magnetic resonance imaging (fMRI), model quality of general linear models (GLMs) for first-level analysis is rarely assessed. In recent work (Soch et al., 2016: “How to avoid mismodelling in GLM-based fMRI data analysis: cross-validated Bayesian model selection”, NeuroImage, vol. 141, pp. 469-489; DOI: 10.1016/j. neuroimage.2016.07.047), we have introduced cross-validated Bayesian model selection (cvBMS) to infer the best model for a group of subjects and use it to guide second-level analysis. While this is the optimal approach given that the same GLM has to be used for all subjects, there is a much more efficient procedure when model selection only addresses nuisance variables and regressors of interest are included in all candidate models. In this work, we propose cross-validated Bayesian model averaging (cvBMA) to improve parameter estimates for these regressors of interest by combining information from all models using their posterior probabilities. This is particularly useful as different models can lead to different conclusions regarding experimental effects and the most complex model is not necessarily the best choice. We find that cvBMS can prevent not detecting established effects and that cvBMA can be more sensitive to experimental effects than just using even the best model in each subject or the model which is best in a group of subjects.

scholarly journals Bayesian model averaging to explore the worth of data for soil‐plant model selection and prediction

2015 ◽  
Vol 51 (4) ◽  
pp. 2825-2846 ◽  
Author(s):  
Thomas Wöhling ◽  
Anneli Schöniger ◽  
Sebastian Gayler ◽  
Wolfgang Nowak
Keyword(s):  
Model Selection ◽  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Plant Model

scholarly journals An introduction to thermodynamic integration and application to dynamic causal models

2021 ◽  
Author(s):  
Eduardo A. Aponte ◽  
Yu Yao ◽  
Sudhir Raman ◽  
Stefan Frässle ◽  
Jakob Heinzle ◽  
...  
Keyword(s):  
Model Selection ◽  
Bayesian Model ◽  
Statistical Physics ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Bayesian Model Selection ◽  
Monte Carlo Sampling ◽  
Thermodynamic Integration ◽  
Causal Modeling ◽  
Variational Bayes

AbstractIn generative modeling of neuroimaging data, such as dynamic causal modeling (DCM), one typically considers several alternative models, either to determine the most plausible explanation for observed data (Bayesian model selection) or to account for model uncertainty (Bayesian model averaging). Both procedures rest on estimates of the model evidence, a principled trade-off between model accuracy and complexity. In the context of DCM, the log evidence is usually approximated using variational Bayes. Although this approach is highly efficient, it makes distributional assumptions and is vulnerable to local extrema. This paper introduces the use of thermodynamic integration (TI) for Bayesian model selection and averaging in the context of DCM. TI is based on Markov chain Monte Carlo sampling which is asymptotically exact but orders of magnitude slower than variational Bayes. In this paper, we explain the theoretical foundations of TI, covering key concepts such as the free energy and its origins in statistical physics. Our aim is to convey an in-depth understanding of the method starting from its historical origin in statistical physics. In addition, we demonstrate the practical application of TI via a series of examples which serve to guide the user in applying this method. Furthermore, these examples demonstrate that, given an efficient implementation and hardware capable of parallel processing, the challenge of high computational demand can be overcome successfully. The TI implementation presented in this paper is freely available as part of the open source software TAPAS.

scholarly journals Thermodynamic integration for dynamic causal models

2018 ◽  
Author(s):  
Eduardo A. Aponte ◽  
Sudhir Raman ◽  
Stefan Frässle ◽  
Jakob Heinzle ◽  
Will D. Penny ◽  
...  
Keyword(s):  
Model Selection ◽  
Bayesian Model ◽  
Model Comparison ◽  
Bayesian Model Averaging ◽  
Model Averaging ◽  
Computation Time ◽  
Thermodynamic Integration ◽  
Causal Modeling ◽  
Single Subject ◽  
Model Evidence

AbstractIn generative modeling of neuroimaging data, such as dynamic causal modeling (DCM), one typically considers several alternative models, either to determine the most plausible explanation for observed data (Bayesian model selection) or to account for model uncertainty (Bayesian model averaging). Both procedures rest on estimates of the model evidence, a principled trade-off between model accuracy and complexity. In DCM, the log evidence is usually approximated using variational Bayes (VB) under the Laplace approximation (VBL). Although this approach is highly efficient, it makes distributional assumptions and can be vulnerable to local extrema. An alternative to VBL is Markov Chain Monte Carlo (MCMC) sampling, which is asymptotically exact but orders of magnitude slower than VB. This has so far prevented its routine use for DCM.This paper makes four contributions. First, we introduce a powerful MCMC scheme – thermodynamic integration (TI) – to neuroimaging and present a derivation that establishes a theoretical link to VB. Second, this derivation is based on a tutorial-like introduction to concepts of free energy in physics and statistics. Third, we present an implementation of TI for DCM that rests on population MCMC. Fourth, using simulations and empirical functional magnetic resonance imaging (fMRI) data, we compare log evidence estimates obtained by TI, VBL, and other MCMC-based estimators (prior arithmetic mean and posterior harmonic mean). We find that model comparison based on VBL gives reliable results in most cases, justifying its use in standard DCM for fMRI. Furthermore, we demonstrate that for complex and/or nonlinear models, TI may provide more robust estimates of the log evidence. Importantly, accurate estimates of the model evidence can be obtained with TI in acceptable computation time. This paves the way for using DCM in scenarios where the robustness of single-subject inference and model selection becomes paramount, such as differential diagnosis in clinical applications.

scholarly journals A Conceptual Introduction to Bayesian Model Averaging

2020 ◽  
Vol 3 (2) ◽  
pp. 200-215
Author(s):  
Max Hinne ◽  
Quentin F. Gronau ◽  
Don van den Bergh ◽  
Eric-Jan Wagenmakers
Keyword(s):  
Model Selection ◽  
Bayesian Model ◽  
Bayesian Model Averaging ◽  
Selection Process ◽  
Process Analysis ◽  
Meta Analysis ◽  
Model Averaging ◽  
Analysis Of Covariance ◽  
Parameter Estimates ◽  
Data Generating Process

Many statistical scenarios initially involve several candidate models that describe the data-generating process. Analysis often proceeds by first selecting the best model according to some criterion and then learning about the parameters of this selected model. Crucially, however, in this approach the parameter estimates are conditioned on the selected model, and any uncertainty about the model-selection process is ignored. An alternative is to learn the parameters for all candidate models and then combine the estimates according to the posterior probabilities of the associated models. This approach is known as Bayesian model averaging (BMA). BMA has several important advantages over all-or-none selection methods, but has been used only sparingly in the social sciences. In this conceptual introduction, we explain the principles of BMA, describe its advantages over all-or-none model selection, and showcase its utility in three examples: analysis of covariance, meta-analysis, and network analysis.

Sign in / Sign up

Export Citation Format

Share Document