Application of a Cognitive Diagnostic Model to a High-Stakes Reading Comprehension Test

General cognitive diagnostic models (CDM) such as the generalized deterministic input, noisy, “and” gate (G-DINA) model are flexible in that they allow for both compensatory and noncompensatory relationships among the subskills within the same test. Most of the previous CDM applications in the literature have been add-ons to simulation studies. Although there are some applications of CDMs such as the Fusion Model and the Rule Space Model to educational assessment data in general and second-language data in particular, there are few studies applying general models such as the G-DINA. The purpose of the present study was to demonstrate the application of the G-DINA to the reading comprehension data of a high-stakes test. To this end, an initial Q-matrix was developed, validated, and cross-validated. The skill profiles of the test takers were estimated using the “CDM” package in R. Throughout, the process of constructing and validating a Q-matrix was elaborated on, the benefits of general models were emphasized, and implications for research investigating inter-skill relationships were discussed. Finally, suggestions for further research, to better take advantage of the flexibilities of general diagnostic models, were presented.

Detection of Misconceptions and Misleading Questions by Using Quantitative Diagnostic Assessment

Researches into E-Learning pay emphasis on how the technology is applied for comprehensive learning support in various perspectives. Lots of advantages are involved while performing the E-Learning. For example, timely contents are applicable to be delivered via the Internet, and in a similar manner, instant feedback which identifies the learning performance would be obtained through an online assessment system. In this article, a diagnostic approach for learning performance evaluation of student is proposed. Two primary issues are concentrated based on the analysis of quantitative assessment results. First the authors identify the misconceptions on specific subject(s) that students have learned and provide remedial suggestions about their learning status to students. Second, because questions may have several incorrect information to students, hence the authors approach provide suggested revisions to instructors. The proposed approach employs the Rule-Space model and the Student-Problem chart and is implemented in an online assessment system to obtain corresponding data (e.g., learning behavior) and performance (e.g., learning status). The feasibility is demonstrated through an empirical study (i.e., a series of pre-post tests) which was conducted in a domain-specific course (i.e., Fundamental of Information Technology) with around 450 students in a calendar year.