scholarly journals PISA, TIMSS and Swedish students’ knowledge of linear equations: A ‘telling’ case of a system fixing something that may not be broken

2020 ◽  
Vol 12 ◽  
pp. 5-29
Author(s):  
Paul Andrews ◽  
Kristina Palm Kaplan
Keyword(s):  
Social Science ◽  
Large Scale ◽  
Linear Equations ◽  
Relational Understanding ◽  
Counter Example ◽  
Case Comparison ◽  
Large Scale Assessments ◽  
Prevailing View ◽  
Case Based

In this paper, we construct a ‘telling’ case to highlight a problematic inconsistency between the results of international large-scale assessments (ILSAs) and other studies of Swedish students’ knowledge of linear equations.In this context, a ‘telling’ case, based on the scrutiny of appropriately chosen cases, is presented as a social science counter-example to the prevailing view that ILSAs’ assessments are not only valid but should underpin systemicreform. Our ‘telling’ case comparison of the different forms of study shows that Swedish students, in contrast with the summative assertions of the different ILSAs, have a secure and relational understanding of linear equationsthat persists into adulthood. We conclude with a cautionary message for the curriculum authorities.

scholarly journals Why ability point estimates can be pointless: a primer on using skill measures from large-scale assessments in secondary analyses

2021 ◽  
Vol 3 (1) ◽  
Author(s):  
Clemens M. Lechner ◽  
Nivedita Bhaktha ◽  
Katharina Groskurth ◽  
Matthias Bluemke
Keyword(s):  
Measurement Error ◽  
Statistical Models ◽  
Test Scores ◽  
Large Scale ◽  
Equation Modeling ◽  
Model Parameters ◽  
Advantages And Disadvantages ◽  
Point Estimates ◽  
Secondary Analyses ◽  
Large Scale Assessments

AbstractMeasures of cognitive or socio-emotional skills from large-scale assessments surveys (LSAS) are often based on advanced statistical models and scoring techniques unfamiliar to applied researchers. Consequently, applied researchers working with data from LSAS may be uncertain about the assumptions and computational details of these statistical models and scoring techniques and about how to best incorporate the resulting skill measures in secondary analyses. The present paper is intended as a primer for applied researchers. After a brief introduction to the key properties of skill assessments, we give an overview over the three principal methods with which secondary analysts can incorporate skill measures from LSAS in their analyses: (1) as test scores (i.e., point estimates of individual ability), (2) through structural equation modeling (SEM), and (3) in the form of plausible values (PVs). We discuss the advantages and disadvantages of each method based on three criteria: fallibility (i.e., control for measurement error and unbiasedness), usability (i.e., ease of use in secondary analyses), and immutability (i.e., consistency of test scores, PVs, or measurement model parameters across different analyses and analysts). We show that although none of the methods are optimal under all criteria, methods that result in a single point estimate of each respondent’s ability (i.e., all types of “test scores”) are rarely optimal for research purposes. Instead, approaches that avoid or correct for measurement error—especially PV methodology—stand out as the method of choice. We conclude with practical recommendations for secondary analysts and data-producing organizations.

Sign in / Sign up

Export Citation Format

Share Document