Apples, oranges, and ironing boards: Comparative effect sizes influence lay impressions of test validity

2021 ◽  
Author(s):  
Marie Childers ◽  
Scott Highhouse ◽  
Margaret E. Brooks
Keyword(s):  
Test Validity ◽  
Effect Sizes ◽  
Comparative Effect

scholarly journals Comparative effect sizes in randomised trials from less developed and more developed countries: meta-epidemiological assessment

BMJ ◽  
2013 ◽  
Vol 346 (feb12 1) ◽  
pp. f707-f707 ◽  
Author(s):  
O. A. Panagiotou ◽  
D. G. Contopoulos-Ioannidis ◽  
J. P. A. Ioannidis
Keyword(s):  
Developed Countries ◽  
Effect Sizes ◽  
Randomised Trials ◽  
Comparative Effect

Factors Influencing Cognate Performance for Young Multilingual Children's Vocabulary: A Research Synthesis

2020 ◽  
Vol 29 (4) ◽  
pp. 2170-2188
Author(s):  
Lindsey R. Squires ◽  
Sara J. Ohlfest ◽  
Kristen E. Santoro ◽  
Jennifer L. Roberts
Keyword(s):  
Test Validity ◽  
Reaction Times ◽  
Bilingual Children ◽  
Second Grade ◽  
High Quality Research ◽  
Word Level ◽  
Research Questions ◽  
Word Characteristics ◽  
Study Inclusion ◽  
Task Level

Purpose The purpose of this systematic review was to determine evidence of a cognate effect for young multilingual children (ages 3;0–8;11 [years;months], preschool to second grade) in terms of task-level and child-level factors that may influence cognate performance. Cognates are pairs of vocabulary words that share meaning with similar phonology and/or orthography in more than one language, such as rose – rosa (English–Spanish) or carrot – carotte (English–French). Despite the cognate advantage noted with older bilingual children and bilingual adults, there has been no systematic examination of the cognate research in young multilingual children. Method We conducted searches of multiple electronic databases and hand-searched article bibliographies for studies that examined young multilingual children's performance with cognates based on study inclusion criteria aligned to the research questions. Results The review yielded 16 articles. The majority of the studies (12/16, 75%) demonstrated a positive cognate effect for young multilingual children (measured in higher accuracy, faster reaction times, and doublet translation equivalents on cognates as compared to noncognates). However, not all bilingual children demonstrated a cognate effect. Both task-level factors (cognate definition, type of cognate task, word characteristics) and child-level factors (level of bilingualism, age) appear to influence young bilingual children's performance on cognates. Conclusions Contrary to early 1990s research, current researchers suggest that even young multilingual children may demonstrate sensitivity to cognate vocabulary words. Given the limits in study quality, more high-quality research is needed, particularly to address test validity in cognate assessments, to develop appropriate cognate definitions for children, and to refine word-level features. Only one study included a brief instruction prior to assessment, warranting cognate treatment studies as an area of future need. Supplemental Material https://doi.org/10.23641/asha.12753179

Abstract Semantic Associative Network Training: A Replication and Update of an Abstract Word Retrieval Therapy Program

2020 ◽  
Vol 29 (3) ◽  
pp. 1574-1595
Author(s):  
Chaleece W. Sandberg ◽  
Teresa Gray
Keyword(s):  
Concrete Word ◽  
Abstract Word ◽  
Distinct Pattern ◽  
Word Retrieval ◽  
Effect Sizes ◽  
Associative Network ◽  
Direct Training ◽  
Network Training ◽  
Abstract Words ◽  
Concrete Words

Purpose We report on a study that replicates previous treatment studies using Abstract Semantic Associative Network Training (AbSANT), which was developed to help persons with aphasia improve their ability to retrieve abstract words, as well as thematically related concrete words. We hypothesized that previous results would be replicated; that is, when abstract words are trained using this protocol, improvement would be observed for both abstract and concrete words in the same context-category, but when concrete words are trained, no improvement for abstract words would be observed. We then frame the results of this study with the results of previous studies that used AbSANT to provide better evidence for the utility of this therapeutic technique. We also discuss proposed mechanisms of AbSANT. Method Four persons with aphasia completed one phase of concrete word training and one phase of abstract word training using the AbSANT protocol. Effect sizes were calculated for each word type for each phase. Effect sizes for this study are compared with the effect sizes from previous studies. Results As predicted, training abstract words resulted in both direct training and generalization effects, whereas training concrete words resulted in only direct training effects. The reported results are consistent across studies. Furthermore, when the data are compared across studies, there is a distinct pattern of the added benefit of training abstract words using AbSANT. Conclusion Treatment for word retrieval in aphasia is most often aimed at concrete words, despite the usefulness and pervasiveness of abstract words in everyday conversation. We show the utility of AbSANT as a means of improving not only abstract word retrieval but also concrete word retrieval and hope this evidence will help foster its application in clinical practice.

Clinically Meaningful Change

Methodology ◽  
2019 ◽  
Vol 15 (3) ◽  
pp. 97-105
Author(s):  
Rodrigo Ferrer ◽  
Antonio Pardo
Keyword(s):  
Effect Size ◽  
False Negative ◽  
False Negative Rate ◽  
Point Of View ◽  
Skewed Distribution ◽  
Effect Sizes ◽  
False Negatives ◽  
Large Size ◽  
Before And After ◽  
Post Test

Abstract. In a recent paper, Ferrer and Pardo (2014) tested several distribution-based methods designed to assess when test scores obtained before and after an intervention reflect a statistically reliable change. However, we still do not know how these methods perform from the point of view of false negatives. For this purpose, we have simulated change scenarios (different effect sizes in a pre-post-test design) with distributions of different shapes and with different sample sizes. For each simulated scenario, we generated 1,000 samples. In each sample, we recorded the false-negative rate of the five distribution-based methods with the best performance from the point of view of the false positives. Our results have revealed unacceptable rates of false negatives even with effects of very large size, starting from 31.8% in an optimistic scenario (effect size of 2.0 and a normal distribution) to 99.9% in the worst scenario (effect size of 0.2 and a highly skewed distribution). Therefore, our results suggest that the widely used distribution-based methods must be applied with caution in a clinical context, because they need huge effect sizes to detect a true change. However, we made some considerations regarding the effect size and the cut-off points commonly used which allow us to be more precise in our estimates.

Test-Taking Motivation and Personality Test Validity

2010 ◽  
Vol 9 (3) ◽  
pp. 117-125 ◽  
Author(s):  
Thomas A. O’Neill ◽  
Richard D. Goffin ◽  
Ian R. Gellatly
Keyword(s):  
Personality Assessment ◽  
Personnel Selection ◽  
Test Validity ◽  
Personality Testing ◽  
Personality Test ◽  
Field Sample ◽  
Moderation Effect ◽  
Test Taking ◽  
Low Levels ◽  
Valid Criterion

In this study we assessed whether the predictive validity of personality scores is stronger when respondent test-taking motivation (TTM) is higher rather than lower. Results from a field sample comprising 269 employees provided evidence for this moderation effect for one trait, Steadfastness. However, for Conscientiousness, valid criterion prediction was only obtained at low levels of TTM. Thus, it appears that TTM relates to the criterion validity of personality testing differently depending on the personality trait assessed. Overall, these and additional findings regarding the nomological net of TTM suggest that it is a unique construct that may have significant implications when personality assessment is used in personnel selection.

The Limits of Ego Depletion

2019 ◽  
Vol 50 (5-6) ◽  
pp. 292-304 ◽  
Author(s):  
Mario Wenzel ◽  
Marina Lind ◽  
Zarah Rowland ◽  
Daniela Zahn ◽  
Thomas Kubiak
Keyword(s):  
Research Agenda ◽  
Statistical Power ◽  
Interindividual Variability ◽  
Ego Depletion ◽  
Crossover Design ◽  
Effect Sizes ◽  
Future Research ◽  
Future Research Agenda ◽  
Depletion Effect ◽  
Within Subjects

Abstract. Evidence on the existence of the ego depletion phenomena as well as the size of the effects and potential moderators and mediators are ambiguous. Building on a crossover design that enables superior statistical power within a single study, we investigated the robustness of the ego depletion effect between and within subjects and moderating and mediating influences of the ego depletion manipulation checks. Our results, based on a sample of 187 participants, demonstrated that (a) the between- and within-subject ego depletion effects only had negligible effect sizes and that there was (b) large interindividual variability that (c) could not be explained by differences in ego depletion manipulation checks. We discuss the implications of these results and outline a future research agenda.

Sign in / Sign up

Export Citation Format

Share Document