Robust variance estimation in meta-regression with dependent effect size estimates

2010 ◽  
Vol 1 (1) ◽  
pp. 39-65 ◽  
Author(s):  
Larry V. Hedges ◽  
Elizabeth Tipton ◽  
Matthew C. Johnson
Keyword(s):  
Effect Size ◽  
Variance Estimation ◽  
Dependent Effect ◽  
Robust Variance ◽  
Robust Variance Estimation ◽  
Meta Regression ◽  
Size Estimates

scholarly journals Meta-Analysis with Robust Variance Estimation: Expanding the Range of Working Models

2020 ◽  
Author(s):  
James E Pustejovsky ◽  
Elizabeth Tipton
Keyword(s):  
Variance Estimation ◽  
Meta Analysis ◽  
Estimation Methods ◽  
Single Type ◽  
Dependence Structure ◽  
Dependent Effect ◽  
Working Models ◽  
Robust Variance ◽  
Robust Variance Estimation ◽  
Meta Regression

In prevention science and related fields, large meta-analyses are common, and these analyses often involve dependent effect size estimates. Robust variance estimation (RVE) methods provide a way to include all dependent effect sizes in a single meta-regression model, even when the nature of the dependence is unknown. RVE uses a working model of the dependence structure, but the two currently available working models are limited to each describing a single type of dependence. Drawing on flexible tools from multivariate meta-analysis, this paper describes an expanded range of working models, along with accompanying estimation methods, which offer benefits in terms of better capturing the types of data structures that occur in practice and improving the efficiency of meta-regression estimates. We describe how the methods can be implemented using existing software (the ‘metafor’ and ‘clubSandwich’ packages for R) and illustrate the approach in a meta-analysis of randomized trials examining the effects of brief alcohol interventions for adolescents and young adults.

scholarly journals Multi-level meta-analysis of single-case experimental designs using robust variance estimation

2021 ◽  
Author(s):  
Man Chen ◽  
James E Pustejovsky
Keyword(s):  
Effect Size ◽  
Variance Estimation ◽  
Single Case ◽  
Meta Analysis ◽  
Data Series ◽  
Robust Variance ◽  
Auto Correlation ◽  
Robust Variance Estimation ◽  
Multi Level ◽  
Size Estimates

Single-case experimental designs (SCEDs) are used to study the effects of interventions on the behavior of individual cases, by making comparisons between repeated measurements of an outcome under different conditions. In research areas where SCEDs are prevalent, there is a need for methods to synthesize results across multiple studies. One approach to synthesis uses a multi-level meta-analysis (MLMA) model to describe the distribution of effect sizes across studies and across cases within studies. However, MLMA relies on having accurate sampling variances of effect size estimates for each case, which may not be possible due to auto-correlation in the raw data series. One possible solution is to combine MLMA with robust variance estimation (RVE), which provides valid assessments of uncertainty even if the sampling variances of effect size estimates are inaccurate. Another possible solution is to forgo MLMA and use simpler, ordinary least squares (OLS) methods, with RVE. This study evaluates the performance of effect size estimators and methods of synthesizing SCEDs in the presence of auto-correlation, for several different effect size metrics, via a Monte Carlo simulation designed to emulate the features of real data series. Results demonstrate that the MLMA model with RVE performs properly in terms of bias, accuracy, and confidence interval coverage for estimating overall average log response ratios. The OLS estimator corrected with RVE performs the best in estimating overall average Tau effect sizes. None of the available methods perform adequately for meta-analysis of within-case standardized mean differences.

scholarly journals An assessment of statistical methods for non-independent data in ecological meta-analyses: Comment

2021 ◽  
Author(s):  
Shinichi Nakagawa ◽  
Alistair M Senior ◽  
Wolfgang Viechtbauer ◽  
Daniel W.A. Noble
Keyword(s):  
Random Effects ◽  
Effect Size ◽  
Variance Estimation ◽  
Meta Analysis ◽  
Estimation Method ◽  
Random Effects Model ◽  
Step Method ◽  
Robust Variance ◽  
Robust Variance Estimation ◽  
Meta Analyses

Recently, Song et al. (2020) conducted a simulation study using different methods to deal with non-independence resulting from effect sizes originating from the same paper – a common occurrence in ecological meta-analyses. The main methods that were of interest in their simulations were: 1) a standard random-effects model used in combination with a weighted average effect size for each paper (i.e., a two-step method), 2) a standard random-effects model after randomly choosing one effect size per paper, 3) a multilevel (hierarchical) meta-analysis model, modelling paper identity as a random factor, and 4) a meta-analysis making use of a robust variance estimation method. Based on their simulation results, they recommend that meta-analysts should either use the two-step method, which involves taking a weighted paper mean followed by analysis with a random-effects model, or the robust variance estimation method.

Sign in / Sign up

Export Citation Format

Share Document