Digital Module 21: Results Reporting for Large‐Scale Assessments

2020 ◽  
Vol 39 (4) ◽  
pp. 137-138
Author(s):  
Francis O'Donnell ◽  
April L. Zenisky
Keyword(s):  
Large Scale ◽  
Results Reporting ◽  
Large Scale Assessments ◽  
Digital Module

scholarly journals Why ability point estimates can be pointless: a primer on using skill measures from large-scale assessments in secondary analyses

2021 ◽  
Vol 3 (1) ◽  
Author(s):  
Clemens M. Lechner ◽  
Nivedita Bhaktha ◽  
Katharina Groskurth ◽  
Matthias Bluemke
Keyword(s):  
Measurement Error ◽  
Statistical Models ◽  
Test Scores ◽  
Large Scale ◽  
Equation Modeling ◽  
Model Parameters ◽  
Advantages And Disadvantages ◽  
Point Estimates ◽  
Secondary Analyses ◽  
Large Scale Assessments

AbstractMeasures of cognitive or socio-emotional skills from large-scale assessments surveys (LSAS) are often based on advanced statistical models and scoring techniques unfamiliar to applied researchers. Consequently, applied researchers working with data from LSAS may be uncertain about the assumptions and computational details of these statistical models and scoring techniques and about how to best incorporate the resulting skill measures in secondary analyses. The present paper is intended as a primer for applied researchers. After a brief introduction to the key properties of skill assessments, we give an overview over the three principal methods with which secondary analysts can incorporate skill measures from LSAS in their analyses: (1) as test scores (i.e., point estimates of individual ability), (2) through structural equation modeling (SEM), and (3) in the form of plausible values (PVs). We discuss the advantages and disadvantages of each method based on three criteria: fallibility (i.e., control for measurement error and unbiasedness), usability (i.e., ease of use in secondary analyses), and immutability (i.e., consistency of test scores, PVs, or measurement model parameters across different analyses and analysts). We show that although none of the methods are optimal under all criteria, methods that result in a single point estimate of each respondent’s ability (i.e., all types of “test scores”) are rarely optimal for research purposes. Instead, approaches that avoid or correct for measurement error—especially PV methodology—stand out as the method of choice. We conclude with practical recommendations for secondary analysts and data-producing organizations.

Sign in / Sign up

Export Citation Format

Share Document