Evaluating the Bookmark Judgments of Standard-Setting Panelists

The purpose of this study is to describe a new approach for evaluating the judgments of standard-setting panelists within the context of the bookmark procedure. The bookmark procedure is widely used for setting performance standards on high-stakes assessments. A many-faceted Rasch (MFR) model is proposed for evaluating the bookmark judgments of the panelists, and its use illustrated with standard-setting judgments from the Michigan Educational Assessment Program (MEAP). Panelists set three performance standards to create four performance levels (Apprentice, Basic, Met, and Exceeded). The content area used to illustrate the model is mathematics in Grade 3. The analyses suggest that the MFR model provides a promising framework for examining bookmark judgments.