scholarly journals Does Rearranging Multiple‐Choice Item Response Options Affect Item and Test Performance?

2019 ◽  
Vol 2019 (1) ◽  
pp. 1-14
Author(s):  
Lin Wang
Keyword(s):  
Item Response ◽  
Test Performance ◽  
Multiple Choice ◽  
Response Options ◽  
Multiple Choice Item

On the Impact of the Response Options’ Position on Item Difficulty in Multiple-Choice-Items

2020 ◽  
pp. 1-10
Author(s):  
Bettina Hagenmüller
Keyword(s):  
Large Scale ◽  
Item Difficulty ◽  
Multiple Choice ◽  
Test Construction ◽  
Test Model ◽  
Group Setting ◽  
Response Options ◽  
Large Scale Assessment ◽  
Multiple Choice Item ◽  
The Impact

Abstract. The multiple-choice item format is widely used in test construction and Large-Scale Assessment. So far, there has been little research on the impact of the position of the solution among the response options and the few existing results are even inconsistent. Since it would be an easy way to create parallel items for group setting by altering the response options, the influence of the response options’ position on item difficulty should be examined. The Linear Logistic Test Model ( Fischer, 1972 ) was used to analyze the data of 829 students aged 8–20 years, who worked on general knowledge items. It was found that the position of the solution among the response options has an influence on item difficulty. Items are easiest when the solution is in first place and more difficult when the solution is placed in a middle position or at the end of the set of response options.

多肢選択式項目の項目形式が文法テストパフォーマンスに与える影響について - The Effects of Multiple-Choice Item Formats on Grammar Test Performance

2010 ◽  
Vol 32 (2) ◽  
pp. 169
Author(s):  
孝昭 熊澤
Keyword(s):  
University Students ◽  
Test Performance ◽  
Variance Components ◽  
Multiple Choice ◽  
First Year ◽  
Generalizability Coefficient ◽  
Multiple Choice Item ◽  
Research Questions ◽  
Generalizability Study ◽  
First Year University

Item formats are a facet of testing that influences examinees’ test performance. In this study, six types of item formats were adopted, and 55 multiple-choice grammar items were developed and administered to 608 first-year university students for placement and diagnostic purposes. The research questions were: to what extent do items function for placement and diagnostic purposes, to what extent do item formats differ in terms of difficulty, to what extent do item formats contribute to the total score variance, and to what extent is the grammar test reliable. Based on the item analyses, most items functioned for placement and diagnostic purposes. FACETS analysis revealed that the six item formats differed in terms of difficulty. The generalizability study showed that 3% of the variance components was due to the item formats. The decision study indicated that the generalizability coefficient and dependability index were satisfactory for placement and diagnostic purposes. Based on the results, implications are discussed. 項目形式はテストパフォーマンスに影響する一つの要因だといわれている。本研究では6種類の項目形式を用いて55項目の多肢選択式項目を文法テストとして作成し、608名の大学一年生にプレイスメントテストと診断テストとして実施した結果を報告する。研究目的は、項目の機能、項目形式の困難度、項目形式などの分散成分の割合、文法テストの信頼性の検証である。項目分析の結果、ほとんどの項目がプレイスメントと診断テストの項目として機能していたことがわかった。FACETS分析の結果、6種類の多肢選択式項目の形式はそれぞれ困難度が異なることがわかった。一般化可能性研究の結果、項目形式の違いによって生じた分散成分が若干あり、文法テストパフォーマンスに影響する要因であることも明らかとなった。決定研究の結果、一般化可能性係数は.81で、信頼度指数は.75であった。上述の結果を踏まえ、教育的示唆について論じた。

Sequence in multiple choice item options.

1952 ◽  
Vol 43 (6) ◽  
pp. 364-368 ◽  
Author(s):  
Scarvia B. Anderson
Keyword(s):  
Multiple Choice ◽  
Multiple Choice Item

A Natural-Language-Processing-Based Procedure for Generating Distractors for Multiple-Choice Questions

2021 ◽  
pp. 016327872110469
Author(s):  
Peter Baldwin ◽  
Janet Mee ◽  
Victoria Yaneva ◽  
Miguel Paniagua ◽  
Jean D’Angelo ◽  
...  
Keyword(s):  
Natural Language Processing ◽  
Natural Language ◽  
Language Processing ◽  
Multiple Choice ◽  
Incorrect Response ◽  
The United States ◽  
Choice Test ◽  
Multiple Choice Questions ◽  
Response Options ◽  
Medical Licensing

One of the most challenging aspects of writing multiple-choice test questions is identifying plausible incorrect response options—i.e., distractors. To help with this task, a procedure is introduced that can mine existing item banks for potential distractors by considering the similarities between a new item’s stem and answer and the stems and response options for items in the bank. This approach uses natural language processing to measure similarity and requires a substantial pool of items for constructing the generating model. The procedure is demonstrated with data from the United States Medical Licensing Examination (USMLE®). For about half the items in the study, at least one of the top three system-produced candidates matched a human-produced distractor exactly; and for about one quarter of the items, two of the top three candidates matched human-produced distractors. A study was conducted in which a sample of system-produced candidates were shown to 10 experienced item writers. Overall, participants thought about 81% of the candidates were on topic and 56% would help human item writers with the task of writing distractors.

Sign in / Sign up

Export Citation Format

Share Document