Robust variance estimation with dependent effect sizes: practical considerations including a software tutorial in Stata and spss

2013 ◽  
Vol 5 (1) ◽  
pp. 13-30 ◽  
Author(s):  
Emily E. Tanner-Smith ◽  
Elizabeth Tipton
Keyword(s):  
Variance Estimation ◽  
Effect Sizes ◽  
Dependent Effect ◽  
Robust Variance ◽  
Dependent Effect Sizes ◽  
Robust Variance Estimation

scholarly journals Power Approximations for Meta-Analysis of Dependent Effect Sizes

2022 ◽  
Author(s):  
Mikkel Helding Vembye ◽  
James E Pustejovsky ◽  
Terri Pigott
Keyword(s):  
Variance Estimation ◽  
Type I Error ◽  
Meta Analysis ◽  
A Priori ◽  
Effect Sizes ◽  
Type I ◽  
Dependent Effect ◽  
Dependent Effect Sizes ◽  
Robust Variance Estimation ◽  
Analytic Models

Meta-analytic models for dependent effect sizes have grown increasingly sophisticated over the last few decades, which has created challenges for a priori power calculations. We introduce power approximations for tests of average effect sizes based upon the most common models for handling dependent effect sizes. In a Monte Carlo simulation, we show that the new power formulas can accurately approximate the true power of common meta-analytic models for dependent effect sizes. Lastly, we investigate the Type I error rate and power for several common models, finding that tests using robust variance estimation provide better Type I error calibration than tests with model-based variance estimation. We consider implications for practice with respect to selecting a working model and an inferential approach.

scholarly journals Meta-Analysis with Robust Variance Estimation: Expanding the Range of Working Models

2020 ◽  
Author(s):  
James E Pustejovsky ◽  
Elizabeth Tipton
Keyword(s):  
Variance Estimation ◽  
Meta Analysis ◽  
Estimation Methods ◽  
Single Type ◽  
Dependence Structure ◽  
Dependent Effect ◽  
Working Models ◽  
Robust Variance ◽  
Robust Variance Estimation ◽  
Meta Regression

In prevention science and related fields, large meta-analyses are common, and these analyses often involve dependent effect size estimates. Robust variance estimation (RVE) methods provide a way to include all dependent effect sizes in a single meta-regression model, even when the nature of the dependence is unknown. RVE uses a working model of the dependence structure, but the two currently available working models are limited to each describing a single type of dependence. Drawing on flexible tools from multivariate meta-analysis, this paper describes an expanded range of working models, along with accompanying estimation methods, which offer benefits in terms of better capturing the types of data structures that occur in practice and improving the efficiency of meta-regression estimates. We describe how the methods can be implemented using existing software (the ‘metafor’ and ‘clubSandwich’ packages for R) and illustrate the approach in a meta-analysis of randomized trials examining the effects of brief alcohol interventions for adolescents and young adults.

Listening Ears or Reading Eyes: A Meta-Analysis of Reading and Listening Comprehension Comparisons

2021 ◽  
pp. 003465432110608
Author(s):  
Virginia Clinton-Lisell
Keyword(s):  
Listening Comprehension ◽  
Variance Estimation ◽  
Meta Analysis ◽  
Age Groups ◽  
Robust Variance ◽  
Reading Condition ◽  
P 16 ◽  
Robust Variance Estimation ◽  
Literal Comprehension

In this study, a meta-analysis of reading and listening comprehension comparisons across age groups was conducted. Based on robust variance estimation (46 studies; N = 4,687), the overall difference between reading and listening comprehension was not reliably different (g = 0.07, p = .23). Reading was beneficial over listening when the reading condition was self-paced (g = 0.13, p = .049) rather than experimenter-paced (g = −0.32, p = .16). Reading also had a benefit when inferential and general comprehension rather than literal comprehension was assessed (g = 0.36, p = .02; g = .15, p = .05; g = −0.01, p = .93, respectively). There was some indication that reading and listening were more similar in languages with transparent orthographies than opaque orthographies (g = 0.001, p = .99; g = 0.10, p = .19, respectively). The findings may be used to inform theories of comprehension about modality influences in that both lower-level skill and affordances vary comparisons of reading and listening comprehension. Moreover, the findings may guide choices of modality; however, both audio and written options are needed for accessible instruction.

Sign in / Sign up

Export Citation Format

Share Document