scholarly journals The versatility of generalizability theory as a tool for exploring and controlling measurement error

2014 ◽  
Vol 31 (2) ◽  
pp. 55-73 ◽  
Author(s):  
Sandra Johnson
Keyword(s):  
Measurement Error ◽  
Error Estimation ◽  
Generalizability Theory ◽  
Educational Assessment ◽  
Measurement Procedure ◽  
Estimation And Prediction ◽  
Generalizability Analysis ◽  
Many Sources

Measurement error arises from many sources in educational assessment. It is important to estimate the importance of this error, and, if appropriate, to seek ways to reduce it. Generalizability theory represents a powerful tool in this sense, allowing identifiable error contributions to be separately quantified, and measurement error to be estimated and even predicted in response to possible changes in the measurement procedure. The paper offers examples of generalizability analysis of numeracy attainment data deriving from the Scottish Survey of Achievement, with the aim of illustrating the versatility of the methodology for error estimation and prediction in this type of sample-based programme.

scholarly journals Estimasi kesalahan pengukuran standard setting dalam penilaian kompetensi matematika tingkat SMP di Kabupaten Sumbawa

2018 ◽  
Vol 22 (1) ◽  
pp. 24-34
Author(s):  
Weni Wendari ◽  
Samsul Hadi
Keyword(s):  
Measurement Error ◽  
Focus Group ◽  
Error Estimation ◽  
Focus Group Discussion ◽  
Generalizability Theory ◽  
Group Discussion ◽  
Standard Setting ◽  
Group Method ◽  
Theory Approach ◽  
Cut Score

Penelitian ini bertujuan untuk mengetahui metode yang lebih akurat dalam mengestimasi kesalahan pengukuran standard setting pada metode Ebel, Bookmark, dan Contrasting group. Data penelitian ini merupakan dokumen Dinas Pendidikan dan Kebudayaan Kabupaten Sumbawa berupa respon peserta Ujian Nasional Matematika Paket P0C5520 tahun ajaran 2015/2016 yang berjumlah 352 siswa. Guru juga dilibatkan dalam penelitian sebagai panelis dalam Focus Group Discussion  (FGD). Data yang terkumpul kemudian dianalisis melaui tiga tahap.. Tahap pertama yaitu persiapan, kegiatan pada tahap ini mencakup penyiapan data, penggolongan SMP, dan penentuan karakteristik butir. Tahap kedua yaitu FGD dilakukan dalam dua putaran. Tahap ketiga yaitu mengestimasi kesalahan pengukuran dengan menggunakan pendekatan Generalizability Theory dengan bantuan program eduG. Hasil penelitian menunjukkan bahwa metode Contrasting group memiliki estimasi kesalahan pengukuran paling kecil dibandingkan metode Ebel dan Bookmark, oleh karena itu, metode Contrasting grouplebih akurat dibandingkan dengan dua metode lainnya.Kata kunci: cut score, standard setting, generalizability theory MEASUREMENT ERROR ESTIMATION OF STANDARD SETTING IN MATHEMATICS COMPETENCY ASSESSMENT FOR JUNIOR HIGH SCHOOL IN SUMBAWA REGENCYAbstractThis research aims to find the most accurate methods in estimating measurement error of standard setting among Ebel, Bookmark, and Contrasting group methods. The data used in this study were 352 students’ responses on Mathematics National Exam Package P0C5520 in the academic year of 2015/2016. The document was collected from the Department of Education and Culture in Sumbawa Regency. Teachers were also involved in this research as panelists in the Focus Group Discussion (FGD). The data collected were then analyzed through three stages. The first stage was preparation stage, including the activities of data preparation, school classification, and item characteristics analysis. The second stage was two-round FGD. The third stage was estimating the measurement error using Generalizability Theory approach assisted by eduG program. The research result shows that Contrasting Group method produces the smallest measurement error estimation compared to Ebel and Bookmark methods, therefore, Contrasting group method is considered as the most accurate method.Keywords: cut score, standard setting, generalizability theory

scholarly journals How Replicates Can Inform Potential Users of a Measurement Procedure about Measurement Error: Basic Concepts and Methods

Diagnostics ◽  
2021 ◽  
Vol 11 (2) ◽  
pp. 162
Author(s):  
Werner Vach ◽  
Oke Gerke
Keyword(s):  
Measurement Error ◽  
Measurement Procedure ◽  
Statistical Methodology ◽  
Single Measurement ◽  
Conceptual Approach ◽  
Simple Manner ◽  
Scientific Investigations ◽  
Standard Procedures ◽  
Standardized Terminology ◽  
General Statistical

Measurement procedures are not error-free. Potential users of a measurement procedure need to know the expected magnitude of the measurement error in order to justify its use, in particular in health care settings. Gold standard procedures providing exact measurements for comparisons are often lacking. Consequently, scientific investigations of the measurement error are often based on using replicates. However, a standardized terminology (and partially also methodology) for such investigations is lacking. In this paper, we explain the basic conceptual approach of such investigations with minimal reference to existing terminology and describe the link to the existing general statistical methodology. This way, some of the key measures used in such investigations can be explained in a simple manner and some light can be shed on existing terminology. We encourage clearly conceptually distinguishing between investigations of the measurement error of a single measurement procedure and the comparison between different measurement procedures or observers. We also identify an unused potential for more advanced statistical analyses in scientific investigations of the measurement error.

Utility of Analysis of Variance with Repeated Measures Programs for Estimating Reliability

1975 ◽  
Vol 41 (2) ◽  
pp. 441-442 ◽  
Author(s):  
Ronald A. Berk
Keyword(s):  
Measurement Error ◽  
Computer Program ◽  
Analysis Of Variance ◽  
Repeated Measures ◽  
Theory Of Measurement ◽  
Generalizability Theory ◽  
Intraclass Correlation ◽  
Program Output

This paper presents a brief description of the generalizability theory of measurement error and its relation to analysis of variance with repeated measures. The intraclass correlation is defined in terms of analysis of variance computer program output. The usefulness of several programs is discussed.

Sign in / Sign up

Export Citation Format

Share Document