Meta-Analysis of Validity and Review of Alternate Form Reliability and Slope for Curriculum-Based Measurement in Science and Social Studies

Meta-analysis was used to examine curriculum-based measurement in the content areas of social studies and science. Nineteen studies between the years of 1998 and 2020 were reviewed to determine overall mean correlation for criterion validity and examine alternate-form reliability and slope coefficients. An overall mean correlation of .59 was found for criterion validity; however, there was significant heterogeneity across studies suggesting curriculum-based measure (CBM) format or content area may affect findings. Low to high alternative form reliability correlation coefficients were reported across CBM formats between .21 and .89. Studies investigating slopes included mostly vocabulary-matching formats and reported a range from .12 to .65 correct items per week with a mean of .34. Our findings suggest that additional research in the development of these measures in validity, reliability, and slope is warranted.

Technical Characteristics of Curriculum-Based Measurement With Students Who Are Deaf

This study compared the reliability and validity of student scores from paper–pencil and e-based assessments using the “maze” and “silent reading fluency” (SRF) tasks. Forty students who were deaf and hard of hearing and reading between the second and fifth grade reading levels and their teachers (n = 21) participated. For maze, alternate form reliability coefficients obtained from correct scores and correct scores adjusted for guessing ranged from r = .61 to .84 (ps < .01); criterion-related validity coefficients ranged from r = .33 to .67 (most ps < .01). For SRF, reliability coefficients obtained from correct scores ranged from r = .50 to .75 (ps < .01); validity ranged from r = .25 to .72. Differences between student performance on paper–pencil and e-based conditions were generally non-significant for maze; significant differences between conditions for SRF favored the paper–pencil condition. Findings suggest that maze holds promise, with inconclusive results for SRF.

Construction and Examination of Math Subskill Mastery Measures

This study details the construction of parameters for generating subskill mastery math measures to be used for screening, intervention planning, progress monitoring, and proximal program evaluation. Parameters for generating assessment measures were built and tested to verify initial equivalence of generated measures using potential digits correct as a proxy for task difficulty across generated measures. Generated measures met initial equivalence criteria and were subjected to further reliability analysis. Measures were generated and administered 1 week apart at fall and winter to students in Grades K, 1, 3, 5, and 7. Thirty-four screening measures were examined for delayed alternate form reliability, risk decision agreement, and interobserver agreement. Delayed alternate form reliability values generally exceeded r = .80, could be reliably scored, and yielded consistent risk decisions. Future research directions were discussed.

Case Studies Introduction

When we initially proposed a case study track to Erik Bohemia, the ADIM’s 2019 conference chair in London, it was a speculative exercise to ascertain whether an alternate form to the academic paper and the workshop could enrich discourse in the management design community. We are both business-design educators at Parsons School of Design in New York City. Our institution maintains its industry connections principally through our teaching faculty who are drawn from agencies, consultancies and in-house innovation teams globally. Based on our own experience with industry/academic hybridity and through our teaching approach, we developed a hunch that there may be a wide interest in, and applicability [with] this form and we’re gratified for the high level of engagement in this initiative, exemplified by the inaugural set of cases in this volume

An Alternative Form of Feedback

This chapter presents a case study of the use of an alternate form of 360 Feedback at Walmart based on an interview-driven assessment of leader reputation. Following an overview of the changing nature of business requirements for success, the emerging importance of understanding and gathering feedback from highly influential stakeholders with whom leaders may have only limited contact is discussed. The chapter then takes a closer look at Walmart’s existing 360 Feedback process and how that has evolved to include a new secondary measure of leader reputation. The reputational 360 interview measure, which focuses on perceptions of the individual’s leadership style, effectiveness, and ability to influence senior stakeholders, is then presented in detail. Similarities and differences in the process (e.g., types of raters, transparency, and use of results for development) are reviewed, and results and key learnings from the pilot program are offered for those considering a similar approach.