scholarly journals Assessing first-year engineering students' pre-university mathematics knowledge: Preliminary validity results based on an item response theory model

2020 ◽  
Vol 10 (2) ◽  
pp. 259
Author(s):  
Yusuf F. Zakariya ◽  
Hans Kristian Nilsen ◽  
Simon Goodchild ◽  
Kirsten Bjørkestøl
Keyword(s):  
Item Response Theory ◽  
Prior Knowledge ◽  
Item Response ◽  
Engineering Students ◽  
Theory Model ◽  
First Year ◽  
Cognitive Interviews ◽  
Validity And Reliability ◽  
Response Theory ◽  
Mathematics Knowledge

The importance of students’ prior knowledge to their current learning outcomes cannot be overemphasised. Students with adequate prior knowledge are better prepared for the current learning materials than those without the knowledge. However, assessment of engineering students' prior mathematics knowledge has been beset with a lack of uniformity in measuring instruments and inadequate validity studies. This study attempts to provide evidence of validity and reliability of a Norwegian national test of prior mathematics knowledge using an explanatory sequential mixed-methods approach. This approach involves use of an item response theory model followed by cognitive interviews of some students among 201 first-year engineering students that constitute the sample of the study. The findings confirm an acceptable construct validity for the test with reliable items and a high-reliability coefficient of .92 on the whole test. Mixed results are found on discrimination and difficulty indices of questions on the test with some questions having unacceptable discriminations and require improvement, some are easy, and some appear too tricky questions for students. Results from the cognitive interviews reveal the likely reasons for students' difficulty on some questions to be lack of proper understanding of the questions, text misreading, improper grasping of word-problem tasks, and unavailability of calculators. The findings underscore the significance of validity and reliability checks of test instruments and their effect on scoring and computing aggregate scores. The methodological approaches to validity and reliability checks in the present study can be applied to other national contexts.

scholarly journals A Multidimensional Item Response Theory Model for Continuous and Graded Responses With Error in Persons and Items

2021 ◽  
pp. 001316442199841
Author(s):  
Pere J. Ferrando ◽  
David Navarro-González
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Theory Model ◽  
Response Model ◽  
Response Theory ◽  
Continuous Response ◽  
Graded Responses ◽  
Graded Response ◽  
Continuous Responses ◽  
Differential Measurement Error

Item response theory “dual” models (DMs) in which both items and individuals are viewed as sources of differential measurement error so far have been proposed only for unidimensional measures. This article proposes two multidimensional extensions of existing DMs: the M-DTCRM (dual Thurstonian continuous response model), intended for (approximately) continuous responses, and the M-DTGRM (dual Thurstonian graded response model), intended for ordered-categorical responses (including binary). A rationale for the extension to the multiple-content-dimensions case, which is based on the concept of the multidimensional location index, is first proposed and discussed. Then, the models are described using both the factor-analytic and the item response theory parameterizations. Procedures for (a) calibrating the items, (b) scoring individuals, (c) assessing model appropriateness, and (d) assessing measurement precision are finally discussed. The simulation results suggest that the proposal is quite feasible, and an illustrative example based on personality data is also provided. The proposals are submitted to be of particular interest for the case of multidimensional questionnaires in which the number of items per scale would not be enough for arriving at stable estimates if the existing unidimensional DMs were fitted on a separate-scale basis.

Validation study of the Korean version of decent work scale

2021 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Yunsoo Lee ◽  
Ji Hoon Song ◽  
Soo Jung Kim
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Work Engagement ◽  
Item Difficulty ◽  
Test Theory ◽  
Decent Work ◽  
Validity And Reliability ◽  
Response Theory ◽  
Content Type ◽  
Korean Version

Purpose This paper aims to validate the Korean version of the decent work scale and examine the relationship between decent work and work engagement. Design/methodology/approach After completing translation and back translation, the authors surveyed 266 Korean employees from various organizations via network sampling. They assessed Rasch’s model based on item response theory. In addition, they used classical test theory to evaluate the decent work scale’s validity and reliability. Findings The authors found that the current version of the decent work scale has good validity, reliability and item difficulty, and decent work has a positive relationship with work engagement. However, based on item response theory, the assessment showed that three of the items are extremely similar to another item within the same dimension, implying that the items are unable to discriminate among individual traits. Originality/value This study validated the decent work scale in a Korean work environment using Rasch’s (1960) model from the perspective of item response theory.

scholarly journals An item response theory analysis of the Community of Inquiry Scale

2015 ◽  
Vol 16 (2) ◽  
Author(s):  
Mehmet Barış Horzum ◽  
Gülden Kaya Uyanik
Keyword(s):  
Online Learning ◽  
Item Response Theory ◽  
Item Response ◽  
Community Of Inquiry ◽  
Item Analysis ◽  
Classical Test Theory ◽  
Test Theory ◽  
Validity And Reliability ◽  
Response Theory ◽  
Learning Programs

The aim of this study is to examine validity and reliability of Community of Inquiry Scale commonly used in online learning by the means of Item Response Theory. For this purpose, Community of Inquiry Scale version 14 is applied on 1,499 students of a distance education center’s online learning programs at a Turkish state university via internet. The collected data is analyzed by using a statistical software package. Research data is analyzed in three aspects, which are checking model assumptions, checking model-data fit and item analysis. Item and test features of the scale are examined by the means of Graded Response Theory. In order to use this model of IRT, after testing the assumptions out of the data gathered from 1,499 participants, data model compliance was examined. Following the affirmative results gathered from the examinations, all data is analyzed by using GRM. As a result of the study, the Community of Inquiry Scale adapted to Turkish by Horzum (in press) is found to be reliable and valid by the means of Classical Test Theory and Item Response Theory.

On the Validity of Forced Choice Scores Derived From the Thurstonian Item Response Theory Model

Assessment ◽  
2019 ◽  
Vol 27 (4) ◽  
pp. 706-718 ◽  
Author(s):  
Kate E. Walton ◽  
Lina Cherkasova ◽  
Richard D. Roberts
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Discriminant Validity ◽  
Criterion Validity ◽  
Theory Model ◽  
Forced Choice ◽  
Response Theory ◽  
Test Criterion ◽  
Convergent And Discriminant Validity ◽  
Response Biases

Forced choice (FC) measures may be a desirable alternative to single stimulus (SS) Likert items, which are easier to fake and can have associated response biases. However, classical methods of scoring FC measures lead to ipsative data, which have a number of psychometric problems. A Thurstonian item response theory (TIRT) model has been introduced as a way to overcome these issues, but few empirical validity studies have been conducted to ensure its effectiveness. This was the goal of the current three studies, which used FC measures of domains from popular personality frameworks including the Big Five and HEXACO, and both statement and adjective item stems. We computed TIRT and ipsative scores and compared their validity estimates. Convergent and discriminant validity of the scores were evaluated by correlating them with SS scores, and test-criterion validity evidence was evaluated by examining their relationships with meaningful outcomes. In all three studies, there was evidence for the convergent and test-criterion validity of the TIRT scores, though at times this was on par with the validity of the ipsative scores. The discriminant validity of the TIRT scores was problematic and was often worse than the ipsative scores.

Sign in / Sign up

Export Citation Format

Share Document