Limited Use for Limited Data: Incorporating Value-Added Measures as Part of a K-12 Post-Tenure Review Process

2020 ◽  
Author(s):  
Micah Barry
2019 ◽  
Vol 80 (2) ◽  
pp. 365-388
Author(s):  
Joshua Marland ◽  
Matthew Harrick ◽  
Stephen G. Sireci

Student assessment nonparticipation (or opt out) has increased substantially in K-12 schools in states across the country. This increase in opt out has the potential to impact achievement and growth (or value-added) measures used for educator and institutional accountability. In this simulation study, we investigated the extent to which value-added measures of teacher quality are affected as a result of varying degrees of opt out, as well as a result of various types of nonrandom opt out. Results show that the magnitude of opt out and choice of classification scheme has a greater impact on value-added estimates than the type of opt-out patterns simulated in this study. Specifically, root mean square differences in value-added estimates increased as magnitude of opt out increased. In addition, teacher effectiveness classification agreement decreased as opt out magnitude increased. One type of opt out, where the highest achieving students in the highest achieving classrooms opted out, had the largest impact on stability than the other types of opt outs.


2016 ◽  
Vol 32 (5) ◽  
pp. 728-750 ◽  
Author(s):  
Michael P. Brady ◽  
Katie Miller ◽  
Jazarae McCormick ◽  
Lawrence A. Heiser

Educators struggle with “value-added” teacher evaluation models based on high-stakes student assessments. Despite validity and reliability threats, these models evaluate university-based teacher preparation programs (TPPs), and play a role in state and professional accreditation. This study reports a more rational value-added evaluation model linking student performance to teacher candidates’ lessons during Practicum and Student Teaching. Results indicate that K-12 students showed learning gains on these lessons, with mixed findings on comparisons of part-time to full-time internships, academic and functional lessons, and candidates’ grade point averages (GPAs). Results indicated that teacher candidates’ lessons are a viable value-added model (VAM) alternative for TPPs.


2017 ◽  
Vol 119 (4) ◽  
pp. 1-32
Author(s):  
Daniel H. Bowen ◽  
Jonathan N. Mills

Background/Context With a growing body of evidence to support the assertion that teacher quality is vital to producing better student outcomes, policymakers continue to seek solutions to attract and retain the best educators. Performance-based pay is a reform that has become popular in K–12 education over the last decade. This strategy potentially produces positive impacts on student achievement in two ways: better alignment of financial incentives with desired outcomes and improved the composition of the teacher workforce. While evaluations have primarily focused on the former result, there is little research on whether the longer-term implementation of these polices can attract more effective teachers. Purpose In this study we aim to provide evidence for potential long-term impacts that performance-based pay can have on the composition of the teacher workforce by addressing two questions: Does performance-based pay attract fundamentally different individuals, as measured by their risk preferences, to the teaching profession? Are stated preferences for a particular pay format correlated to measures of teacher quality? Research Design We apply methods from experimental economics and conduct surveys with 120 teachers from two school districts who have experienced performance pay. We compare the risk preferences of teachers hired under the two pay formats to test the hypothesis that performance-based pay attracts individuals with different characteristics to the profession. We also analyze teachers’ survey responses on their preferences for performance-based pay to determine their relationships to two measures of teacher quality: student test-score gains and principal evaluations. Conclusions/Recommendations We find mixed results regarding the ability of performance-based pay to alter the composition of the teacher workforce. Teachers hired with performance-based pay in place are no different from their colleagues. However, teachers claiming to seek employment in districts with performance-based pay in place appear significantly less risk averse. Surprisingly, additional analyses indicate that teachers’ value-added scores and performance evaluations do not predict a positive disposition towards merit pay. Thus, while these results indicate the possibility for performance-based pay to attract different individuals to teaching, they do not provide evidence that such change would necessarily improve the composition of the workforce. Policymakers should take this potential tradeoff into consideration when considering the expansion of performance pay policies.


Author(s):  
Thomas C. Hammond ◽  
R. Curby Alexander ◽  
Alec M. Bodzin

The TPACK framework provides researchers with a robust framework for conducting research on technology integration in authentic environments, i.e., intact classrooms engaged in standards-aligned instruction. Researchers who wish to identify the value added by a promising technology-supported instructional strategy will need to assess student learning outcomes in these environments; unfortunately, collecting valid and reliable data on student learning in classroom research is extremely difficult. To date, few studies using TPACK in K-12 classrooms have included student learning outcomes in their research questions, and researchers are therefore left without models to guide their development, implementation, and analysis of assessments. This chapter draws upon the literature and our own research and assessment experiences in technology-integrated, standards-aligned classroom instruction to give examples and advice to researchers as they develop, analyze, and write up their observations of student learning outcomes. In particular, we focus on standard items, specifically multiple choice items, as an accepted (if limited) method for assessing student understanding. We seek to fill an existing gap in the literature between assessment advice for educational psychologists (who typically work outside of classroom settings) and advice given to teachers (who have lower thresholds for issues such as validity and reliability). Classroom researchers will benefit from this advice to develop, validate, and apply their own objective assessments. We focus on the content areas of science and social studies, but this advice can be applied to others as well.


Author(s):  
Clark Shah-Nelson ◽  
Ellen A. Mayo ◽  
Patience Ebuwei

An American K-12 cooperative educational services provider (“The Agency”) has an issue: partner school districts are saving money by building internal capacity for professional development, rather than fully utilizing expertise from the Agency. The aim of this evidence-based case study is to inform the Agency on capacity-building for innovation. The researchers performed three separate rapid evidence assessments, followed by a standard systematic review process to synthesize findings across 31 studies. Key findings identified from the research include (1) organizational capacity and program evaluation lead to organizational sustainability, (2) agency leadership should guide strategic organizational change in order to establish a shared vision for evaluation and feedback, and (3) organizations benefit from practicing continuous and ongoing learning through feedback loops. The findings of this study may be generalizable to other similar educational service providers or non-profits looking to strengthen organizational capacity and partnerships.


2020 ◽  
pp. 1-23
Author(s):  
HALIT YANIKKAYA ◽  
ABDULLAH ALTUN

This study compares the impacts of gross trade openness measures with trade openness in value-added measures on economic growth for the years 1995–2014 by employing a dynamic panel data estimation. Our findings suggest that although gross trade shares promote growth, using value-added trade shares magnifies this positive effect. Compared with gross terms, estimates also imply that while exports in value-added terms have much larger growth effect, imports in value-added terms have no significant impact. We then evaluate the impacts of tariffs on growth in terms of gross trade and trade in value added separately. Although our results imply the negative growth effects of gross import tariffs, this negative impact disappears for tariffs in value-added terms. These results reaffirm that trade protectionism has potential to lower global growth through reducing exports because it is clear that export shares regardless of their measurements and disaggregation levels promote growth. Our results indicate that countries should support not only exports of final products but also exports of intermediates. However, given the necessity of imports for exports, our results do not lend any evidence to discourage overall imports.


2010 ◽  
Vol 24 (3) ◽  
pp. 167-182 ◽  
Author(s):  
Kevin Lang

One of the potential strengths of the No Child Left Behind (NCLB) Act enacted in 2002 is that the law requires the production of an enormous amount of data, particularly from tests, which, if used properly, might help us improve education. As an economist and as someone who served 13 years on the School Committee1 in Brookline Massachusetts, until May 2009, I have been appalled by the limited ability of districts to analyze these data; I have been equally appalled by the cavalier manner in which economists use test scores and related measures in their analyses. The summary data currently provided are very hard to interpret, and policymakers, who typically lack statistical sophistication, cannot easily use them to assess progress. In some domains, most notably the use of average test scores to evaluate teachers or schools, the education community is aware of the biases and has sought better measures. The economics and statistics communities have both responded to and created this demand by developing value-added measures that carry a scientific aura. However, economists have largely failed to recognize many of the problems with such measures. These problems are sufficiently important that they should preclude any automatic link between these measures and rewards or sanctions. They do, however, contain information and can be used as a catalyst for more careful evaluation of teachers and schools, and as a lever to induce principals and other administrators to act on their knowledge.


2018 ◽  
Vol 91 (2) ◽  
pp. 132-158 ◽  
Author(s):  
Paul Hanselman

Are equal educational opportunities sufficient to narrow long-standing economic and racial inequalities in achievement? In this article, I test the hypothesis that poor and minority students benefit less from effective elementary school teachers than do their nonpoor and white peers, thus exacerbating inequalities. I use administrative data from public elementary schools in North Carolina to calculate value-added measures of teachers’ success in promoting learning, and I assess benefits for different students. Results suggest that differential benefits of effective teachers uniquely exacerbate black–white inequalities but do not contribute to economic achievement gaps. Racial differences are small, on average, relative to the benefits for all groups; are not explained by differences in prior achievement; and are largest for low-achieving students. Teacher-related learning opportunities are crucial for all students, but these results point to a disconnect between typical school learning opportunities and low-achieving minority students.


2020 ◽  
Vol 49 (5) ◽  
pp. 335-349
Author(s):  
Allison Atteberry ◽  
Daniel Mangan

Papay (2011) noticed that teacher value-added measures (VAMs) from a statistical model using the most common pre/post testing timeframe–current-year spring relative to previous spring (SS)–are essentially unrelated to those same teachers’ VAMs when instead using next-fall relative to current-fall (FF). This is concerning since this choice–made solely as an artifact of the timing of statewide testing–produces an entirely different ranking of teachers’ effectiveness. Since subsequent studies (grades K/1) have not replicated these findings, we revisit and extend Papay’s analyses in another Grade 3–8 setting. We find similarly low correlations (.13–.15) that persist across value-added specifications. We delineate and apply a literature-based framework for considering the role of summer learning loss in producing these low correlations.


Sign in / Sign up

Export Citation Format

Share Document