Supplemental Material for An Illustration of the Effects of Fluctuations in Test Information on Measurement Error, the Attenuation of Effect Sizes, and Diagnostic Reliability

2017 ◽  
Keyword(s):  
Measurement Error ◽  
Effect Sizes ◽  
Diagnostic Reliability ◽  
Test Information

scholarly journals Determining sexual dimorphism in frog measurement data: integration of statistical significance, measurement error, effect size and biological significance

2005 ◽  
Vol 77 (1) ◽  
pp. 45-76 ◽  
Author(s):  
Lee-Ann C. Hayek ◽  
W. Ronald Heyer
Keyword(s):  
Sexual Dimorphism ◽  
Measurement Error ◽  
Effect Size ◽  
Statistical Significance ◽  
Measurement Data ◽  
Biological Significance ◽  
Effect Sizes ◽  
Statistical Hypothesis ◽  
Multivariate Techniques ◽  
Error Index

Several analytic techniques have been used to determine sexual dimorphism in vertebrate morphological measurement data with no emergent consensus on which technique is superior. A further confounding problem for frog data is the existence of considerable measurement error. To determine dimorphism, we examine a single hypothesis (Ho = equal means) for two groups (females and males). We demonstrate that frog measurement data meet assumptions for clearly defined statistical hypothesis testing with statistical linear models rather than those of exploratory multivariate techniques such as principal components, correlation or correspondence analysis. In order to distinguish biological from statistical significance of hypotheses, we propose a new protocol that incorporates measurement error and effect size. Measurement error is evaluated with a novel measurement error index. Effect size, widely used in the behavioral sciences and in meta-analysis studies in biology, proves to be the most useful single metric to evaluate whether statistically significant results are biologically meaningful. Definitions for a range of small, medium, and large effect sizes specifically for frog measurement data are provided. Examples with measurement data for species of the frog genus Leptodactylus are presented. The new protocol is recommended not only to evaluate sexual dimorphism for frog data but for any animal measurement data for which the measurement error index and observed or a priori effect sizes can be calculated.

scholarly journals On Attenuated Interactions, Measurement Error, and Statistical Power: Guidelines for Social and Personality Psychologists

2020 ◽  
Vol 46 (12) ◽  
pp. 1702-1711 ◽  
Author(s):  
Khandis R. Blake ◽  
Steven Gangestad
Keyword(s):  
Social Psychology ◽  
Measurement Error ◽  
Statistical Power ◽  
Best Practice ◽  
Effect Sizes ◽  
Social Psychology Bulletin ◽  
Replication Crisis ◽  
Positive Results

The replication crisis has seen increased focus on best practice techniques to improve the reliability of scientific findings. What remains elusive to many researchers and is frequently misunderstood is that predictions involving interactions dramatically affect the calculation of statistical power. Using recent papers published in Personality and Social Psychology Bulletin (PSPB), we illustrate the pitfalls of improper power estimations in studies where attenuated interactions are predicted. Our investigation shows why even a programmatic series of six studies employing 2 × 2 designs, with samples exceeding N = 500, can be woefully underpowered to detect genuine effects. We also highlight the importance of accounting for error-prone measures when estimating effect sizes and calculating power, explaining why even positive results can mislead when power is low. We then provide five guidelines for researchers to avoid these pitfalls, including cautioning against the heuristic that a series of underpowered studies approximates the credibility of one well-powered study.

Gender differences in young children’s compliance to maternal directives: A meta-analysis

2019 ◽  
Vol 44 (2) ◽  
pp. 146-156
Author(s):  
Irwin W. Silverman
Keyword(s):  
Gender Differences ◽  
Measurement Error ◽  
Young Children ◽  
Evolutionary Theory ◽  
Meta Analysis ◽  
Effect Sizes ◽  
Positive Relationships ◽  
Lines Of Evidence

Evolutionary theory and several lines of evidence suggest that the motive to establish positive relationships with others is stronger in females than males. Accordingly, it was predicted that in young children, girls would be more likely than boys to comply with their mothers’ directives. To test this prediction, the present meta-analysis examined gender differences in compliance to maternal directives in young children (ages 1–7 years) as assessed on structured tasks. The meta-analysis was performed on 80 effect sizes derived from 49 studies conducted in 10 countries. Two categories of studies were distinguished: those that assessed compliance with respect to the child’s presumed motives for performing compliant and those that assessed compliance without reference to the child’s presumed motives. For the former category of studies, girls were higher in internally motivated compliance whether the task required performing an action or not performing an action, and boys were higher in externally motivated compliance when the task required not performing an action. For the latter category of studies, results were mixed, with some evidence indicating that girls were higher in compliance. No evidence was found indicating that the magnitude of the gender differences changed with age. One caveat is that the effect sizes analyzed might have been attenuated due to measurement error. Discussion focuses on a number of explanations that may be offered to account for the gender differences found in internally motivated compliance favoring girls.

scholarly journals No Evidence Against the Greater Male Variability Hypothesis: A Commentary on Harrison et al.’s Meta-Analysis of Animal Personality

2022 ◽  
Author(s):  
Marco Del Giudice ◽  
Steven Gangestad
Keyword(s):  
Sex Differences ◽  
Measurement Error ◽  
Meta Analysis ◽  
Statistical Significance ◽  
Animal Personality ◽  
Effect Sizes ◽  
Behavioral Measures ◽  
Psychological Traits ◽  
Statistical Validity ◽  
Wide Range

Harrison et al. (2021) set out to test the greater male variability hypothesis with respect to personality in non-human animals. Based on the non-significant results of their meta-analysis, they concluded that there is no evidence to support the hypothesis, and that biological explanations for greater male variability in human psychological traits should be called into question. Here, we show that these conclusions are unwarranted. Specifically: (a) in mammals, birds, and reptiles/amphibians, the magnitude of the sex differences in variability found in the meta-analysis is entirely in line with previous findings from both humans and non-human animals; (b) the generalized lack of statistical significance does not imply that effect sizes were too small to be considered meaningful, as the study was severely underpowered to detect effect sizes in the plausible range; (c) the results of the meta-analysis can be expected to underestimate the true magnitude of sex differences in the variability of personality, because the behavioral measures employed in most of the original studies contain large amounts of measurement error; and (d) variability effect sizes based on personality scores, latencies, and proportions suffer from lac of statistical validity, adding even more noise to the meta-analysis. In total, Harrison et al.’s study does nothing to disprove the greater male variability hypothesis in mammals, let alone in humans. To the extent that they are valid, the data remain compatible with a wide range of plausible scenarios.

Abstract Semantic Associative Network Training: A Replication and Update of an Abstract Word Retrieval Therapy Program

2020 ◽  
Vol 29 (3) ◽  
pp. 1574-1595
Author(s):  
Chaleece W. Sandberg ◽  
Teresa Gray
Keyword(s):  
Concrete Word ◽  
Abstract Word ◽  
Distinct Pattern ◽  
Word Retrieval ◽  
Effect Sizes ◽  
Associative Network ◽  
Direct Training ◽  
Network Training ◽  
Abstract Words ◽  
Concrete Words

Purpose We report on a study that replicates previous treatment studies using Abstract Semantic Associative Network Training (AbSANT), which was developed to help persons with aphasia improve their ability to retrieve abstract words, as well as thematically related concrete words. We hypothesized that previous results would be replicated; that is, when abstract words are trained using this protocol, improvement would be observed for both abstract and concrete words in the same context-category, but when concrete words are trained, no improvement for abstract words would be observed. We then frame the results of this study with the results of previous studies that used AbSANT to provide better evidence for the utility of this therapeutic technique. We also discuss proposed mechanisms of AbSANT. Method Four persons with aphasia completed one phase of concrete word training and one phase of abstract word training using the AbSANT protocol. Effect sizes were calculated for each word type for each phase. Effect sizes for this study are compared with the effect sizes from previous studies. Results As predicted, training abstract words resulted in both direct training and generalization effects, whereas training concrete words resulted in only direct training effects. The reported results are consistent across studies. Furthermore, when the data are compared across studies, there is a distinct pattern of the added benefit of training abstract words using AbSANT. Conclusion Treatment for word retrieval in aphasia is most often aimed at concrete words, despite the usefulness and pervasiveness of abstract words in everyday conversation. We show the utility of AbSANT as a means of improving not only abstract word retrieval but also concrete word retrieval and hope this evidence will help foster its application in clinical practice.

Sign in / Sign up

Export Citation Format

Share Document