Using Lagged Outcomes to Evaluate Bias in Value-Added Models

In this article, the authors provide a methodological critique of the current standard of value-added modeling forwarded in educational policy contexts as a means of measuring teacher effectiveness. Conventional value-added estimates of teacher quality are attempts to determine to what degree a teacher would theoretically contribute, on average, to the test score gains of any student in the accountability population (i.e., district or state). Everson, Feinauer, and Sudweeks suggest an alternative statistical methodology, propensity score matching, which allows estimation of how well a teacher performs relative to teachers assigned comparable classes of students. This approach more closely fits the appropriate role of an accountability system: to estimate how well employees perform in the job to which they are actually assigned. It also has the benefit of requiring fewer statistical assumptions—assumptions that are frequently violated in value-added modeling. The authors conclude that this alternative method allows for more appropriate and policy-relevant inferences about the performance of teachers.

Download Full-text

Estimating Private School Premium for Primary School Children in Ethiopia: Evidence from Individual-level Panel Data

Progress in Development Studies ◽

10.1177/1464993419889708 ◽

2019 ◽

Vol 20 (1) ◽

pp. 26-44

Author(s):

Godstime Osekhebhen Eigbiremolen

Keyword(s):

Public Schools ◽

Private School ◽

Primary Schools ◽

Value Added ◽

Primary School Children ◽

Individual Level ◽

Value Added Models ◽

Value Added Model ◽

Private And Public ◽

Unobserved Ability

This article presents the first value-added model of private school effect in Ethiopia, using the unique Young Lives longitudinal data. I found a substantial and statistically significant private school premium (about 0.5 standard deviation) in Maths, but not in Peabody Picture Vocabulary Test (PPVT). Private school premium works for both low and high ability children. The results are robust to sorting on unobserved ability, grouping on lag structures and transfer between private and public schools. Combined with available contextual data, empirical evidence suggests that the effectiveness of private primary schools may be due to more learning time and teacher’s attention enjoyed by students. I also attempted to contribute methodologically to the literature by directly testing the structural assumption underpinning value-added models.

Download Full-text

Does Student Sorting Invalidate Value-Added Models of Teacher Effectiveness? An Extended Analysis of the Rothstein Critique

Education Finance and Policy ◽

10.1162/edfp_a_00027 ◽

2011 ◽

Vol 6 (1) ◽

pp. 18-42 ◽

Cited By ~ 66

Author(s):

Cory Koedel ◽

Julian R. Betts

Keyword(s):

Teacher Effectiveness ◽

Value Added ◽

Classroom Teaching ◽

First Year ◽

Value Added Models ◽

Value Added Model ◽

Extended Analysis ◽

Research Questions ◽

Value Added Modeling ◽

Student Sorting

Value-added modeling continues to gain traction as a tool for measuring teacher performance. However, recent research questions the validity of the value-added approach by showing that it does not mitigate student-teacher sorting bias (its presumed primary benefit). Our study explores this critique in more detail. Although we find that estimated teacher effects from some value-added models are severely biased, we also show that a sufficiently complex value-added model that evaluates teachers over multiple years reduces the sorting bias problem to statistical insignificance. One implication of our findings is that data from the first year or two of classroom teaching for novice teachers may be insufficient to make reliable judgments about quality. Overall, our results suggest that in some cases value-added modeling will continue to provide useful information about the effectiveness of educational inputs.

Download Full-text

Interpreting Tests of School VAM Validity

The American Economic Review ◽

10.1257/aer.p20161080 ◽

2016 ◽

Vol 106 (5) ◽

pp. 388-392 ◽

Cited By ~ 4

Author(s):

Joshua Angrist ◽

Peter Hull ◽

Parag Pathak ◽

Christopher Walters

Keyword(s):

School District ◽

Predictive Value ◽

Degrees Of Freedom ◽

School Attendance ◽

Value Added ◽

School Effects ◽

Test Statistic ◽

Forecast Bias ◽

Value Added Models

We develop over-identification tests that use admissions lotteries to assess the predictive value of regression-based value-added models (VAMs). These tests have degrees of freedom equal to the number of quasi-experiments available to estimate school effects. By contrast, previously implemented VAM validation strategies look at a single restriction only, sometimes said to measure forecast bias. Tests of forecast bias may be misleading when the test statistic is constructed from many lotteries or quasi-experiments, some of which have weak first stage effects on school attendance. The theory developed here is applied to data from the Charlotte-Mecklenberg School district analyzed by Deming (2014).

Download Full-text