scholarly journals A new approach to assessing shyness of college students using computerized adaptive testing: CAT-Shyness

2020 ◽  
Vol 14 ◽  
Author(s):  
Zifei Li ◽  
Yan Cai ◽  
Dongbo Tu
Keyword(s):  
Psychometric Properties ◽  
Sensitivity And Specificity ◽  
Computerized Adaptive Testing ◽  
Real Data ◽  
Adaptive Testing ◽  
Measurement Precision ◽  
Item Bank ◽  
Test Time ◽  
Local Independence ◽  
Computerized Adaptive Test

Abstract Assessing shy symptoms via computerized adaptive testing (CAT) provides greater measurement precision coupled with a lower test burden compared to conventional tests. The computerized adaptive test for shyness (CAT-Shyness) was developed based on a large sample of 1400 participants from China. Item bank development included the investigation of unidimensionality, local independence, and exploration of differential item functioning (DIF). CAT simulations based on the real data were carried out to investigate the reliability, validity, and predicted utility (sensitivity and specificity) of the CAT-Shyness. The CAT-Shyness item bank was successfully built and proved to have excellent psychometric properties: high content validity, unidimensionality, local independence, and no DIF. The CAT simulations needed 14 items to achieve a high measurement precision with a reliability of .9. Moreover, the results revealed that the proposed CAT-Shyness had acceptable and reasonable marginal reliability, criterion-related validity, and sensitivity and specificity. It not only had acceptable psychometric properties, but also had a shorter but efficient assessment of shyness, which can save significant test time and reduce the test burden for individuals with less information loss.

scholarly journals Measurement Precision and Efficiency of Computerized Adaptive Testing for the Activities-specific Balance Confidence Scale in People With Stroke

2021 ◽  
Author(s):  
Bryant A Seamon ◽  
Steven A Kautz ◽  
Craig A Velozo
Keyword(s):  
Rasch Model ◽  
Computerized Adaptive Testing ◽  
Adaptive Testing ◽  
Measurement Precision ◽  
Strongly Correlated ◽  
Computerized Adaptive Test ◽  
Balance Confidence ◽  
Adaptive Test ◽  
The Rasch Model ◽  
Confidence Scale

Abstract Objective Administrative burden often prevents clinical assessment of balance confidence in people with stroke. A computerized adaptive test (CAT) version of the Activities-specific Balance Confidence Scale (ABC CAT) can dramatically reduce this burden. The objective of this study was to test balance confidence measurement precision and efficiency in people with stroke with an ABC CAT. Methods We conducted a retrospective cross-sectional simulation study with data from 406 adults approximately 2-months post-stroke in the Locomotor-Experience Applied Post-Stroke (LEAPS) trial. Item parameters for CAT calibration were estimated with the Rasch model using a random sample of participants (n = 203). Computer simulation was used with response data from remaining 203 participants to evaluate the ABC CAT algorithm under varying stopping criteria. We compared estimated levels of balance confidence from each simulation to actual levels predicted from the Rasch model (Pearson correlations and mean standard error (SE)). Results Results from simulations with number of items as a stopping criterion strongly correlated with actual ABC scores (full item, r = 1, 12-item, r = 0.994; 8-item, r = 0.98; 4-item, r = 0.929). Mean SE increased with decreasing number of items administered (full item, SE = 0.31; 12-item, SE = 0.33; 8-item, SE = 0.38; 4-item, SE = 0.49). A precision-based stopping rule (mean SE = 0.5) also strongly correlated with actual ABC scores (r = .941) and optimized the relationship between number of items administrated with precision (mean number of items 4.37, range [4–9]). Conclusions An ABC CAT can determine accurate and precise measures of balance confidence in people with stroke with as few as 4 items. Individuals with lower balance confidence may require a greater number of items (up to 9) and attributed to the LEAPS trial excluding more functionally impaired persons. Impact Statement Computerized adaptive testing can drastically reduce the ABC’s test administration time while maintaining accuracy and precision. This should greatly enhance clinical utility, facilitating adoption of clinical practice guidelines in stroke rehabilitation. Lay Summary If you have had a stroke, your physical therapist will likely test your balance confidence. A computerized adaptive test version of the ABC scale can accurately identify balance with as few as 4 questions, which takes much less time.

scholarly journals A computerized adaptive testing advancing the measurement of subjective well-being

2019 ◽  
Vol 13 ◽  
Author(s):  
Yifang Wu ◽  
Yan Cai ◽  
Dongbo Tu
Keyword(s):  
Computerized Adaptive Testing ◽  
Simulated Data ◽  
Real Data ◽  
Well Being ◽  
Adaptive Testing ◽  
Test Time ◽  
Subjective Well Being ◽  
Test Items ◽  
Item Fit ◽  
Item Functioning

AbstractThis article aimed at developing an adaptive version of the subjective well-being (SWB) scale to measure a comprehensive concept of SWB among Chinese university students. Item response theory was employed to formulate the item bank of the SWB scale and computerized adaptive testing (CAT) for SWB (CAT-SWB), based on several commonly used SWB scales, after unidimensionality testing, model selection, local dependence testing, parameter estimation, item fit test and differential item functioning (DIF) analysis were performed. Finally, two CAT simulations using simulated-data and real-data were carried out to verify and evaluate the CAT-SWB. Results indicated that the proposed CAT-SWB had an excellent performance in that it largely reduces the number of test items and the length of test time without losing measurement precision.

An Item Response Theory–Based, Computerized Adaptive Testing Version of the MacArthur–Bates Communicative Development Inventory: Words & Sentences (CDI:WS)

2016 ◽  
Vol 59 (2) ◽  
pp. 281-289 ◽  
Author(s):  
Guido Makransky ◽  
Philip S. Dale ◽  
Philip Havmose ◽  
Dorthe Bleses
Keyword(s):  
Item Response Theory ◽  
Item Response ◽  
Computerized Adaptive Testing ◽  
Real Data ◽  
Adaptive Testing ◽  
Measurement Precision ◽  
Response Theory ◽  
Irt Model ◽  
Communicative Development ◽  
Accuracy And Precision

Purpose This study investigated the feasibility and potential validity of an item response theory (IRT)–based computerized adaptive testing (CAT) version of the MacArthur–Bates Communicative Development Inventory: Words & Sentences (CDI:WS; Fenson et al., 2007) vocabulary checklist, with the objective of reducing length while maintaining measurement precision. Method Parent-reported vocabulary for the American CDI:WS norming sample consisting of 1,461 children between the ages of 16 and 30 months was used to investigate the fit of the items to the 2-parameter logistic IRT model and to simulate CDI-CAT versions with 400, 200, 100, 50, 25, 10, and 5 items. Results All but 14 items fit the 2-parameter logistic IRT model, and real data simulations of CDI-CATs with at least 50 items recovered full CDI scores with correlations over .95. Furthermore, the CDI-CATs with at least 50 items had similar correlations with age and socioeconomic status as the full CDI:WS. Conclusion These results provide strong evidence that a CAT version of the CDI:WS has the potential to reduce length while maintaining the accuracy and precision of the full instrument.

Computerized Adaptive Testing for Sleep Disorders

2021 ◽  
pp. 1-12
Author(s):  
Menghua She ◽  
Yaling Li ◽  
Dongbo Tu ◽  
Yan Cai
Keyword(s):  
Sleep Disorders ◽  
Sensitivity And Specificity ◽  
Computerized Adaptive Testing ◽  
Assessment Tool ◽  
Adaptive Testing ◽  
Item Bank ◽  
Accurate Assessment ◽  
Psychometric Characteristics ◽  
Predictive Utility ◽  
Item Fit

Abstract. Background: As more and more people suffer from sleep disorders, the need to develop an efficient, inexpensive, and accurate assessment tool for screening sleep disorders has become more urgent. Aim: The aim of the current study was to develop a system allowing computerized adaptive testing for sleep disorders (CAT-SD). Methods: A large sample ( N = 1,304) was recruited to construct an item bank for CAT-SD and to investigate the psychometric characteristics of CAT-SD. First, analyses of unidimensionality, model fit, item fit, item discrimination parameters, and differential item functioning (DIF) were conducted to construct a final item pool to meet the requirements of item response theory measurement. Then, a simulated CAT study with real data was performed to investigate the psychometric characteristics of CAT-SD, including the reliability, validity, and predictive utility (sensitivity and specificity). Results: The final unidimensional item bank of the CAT-SD had good item fit, high discrimination, and no DIF. Moreover, it had acceptable reliability, validity, and predictive utility. Limitations: Non-statistical assembly constraints, execution environment, construction of item bank, criterion-related validity, and predictive utility (sensitivity and specificity) of CAT-SD, and sample representativeness are discussed. Conclusions: The CAT-SD could be used as an effective and accurate assessment tool for measuring the sleep disorders in individuals and offers a novel approach to the screening of sleep disorders utilizing psychological scales.

Benefits from Computerized Adaptive Testing as Seen in Simulation Studies

1999 ◽  
Vol 15 (2) ◽  
pp. 91-98 ◽  
Author(s):  
Lutz F. Hornke
Keyword(s):  
Measurement Error ◽  
Computerized Adaptive Testing ◽  
Test Procedure ◽  
Adaptive Testing ◽  
Parameter Estimates ◽  
Simulation Studies ◽  
Computerized Adaptive Test ◽  
Item Banks ◽  
Item Parameters ◽  
General Reliability

Summary: Item parameters for several hundreds of items were estimated based on empirical data from several thousands of subjects. The logistic one-parameter (1PL) and two-parameter (2PL) model estimates were evaluated. However, model fit showed that only a subset of items complied sufficiently, so that the remaining ones were assembled in well-fitting item banks. In several simulation studies 5000 simulated responses were generated in accordance with a computerized adaptive test procedure along with person parameters. A general reliability of .80 or a standard error of measurement of .44 was used as a stopping rule to end CAT testing. We also recorded how often each item was used by all simulees. Person-parameter estimates based on CAT correlated higher than .90 with true values simulated. For all 1PL fitting item banks most simulees used more than 20 items but less than 30 items to reach the pre-set level of measurement error. However, testing based on item banks that complied to the 2PL revealed that, on average, only 10 items were sufficient to end testing at the same measurement error level. Both clearly demonstrate the precision and economy of computerized adaptive testing. Empirical evaluations from everyday uses will show whether these trends will hold up in practice. If so, CAT will become possible and reasonable with some 150 well-calibrated 2PL items.

Computerized Adaptive Testing for Sleep Disorders: Development of An Item Bank and Validation in A Simulated Study

2020 ◽  
Author(s):  
Menghua She ◽  
Yaling Li ◽  
Dongbo Tu ◽  
Yan Cai
Keyword(s):  
Sleep Disorders ◽  
Computerized Adaptive Testing ◽  
Assessment Tool ◽  
Adaptive Testing ◽  
Item Bank ◽  
Accurate Assessment ◽  
Item Pool ◽  
Psychometric Characteristics ◽  
Predictive Utility ◽  
Item Fit

Abstract Background: As more and more people suffer from sleep disorders, developing an efficient, cheap and accurate assessment tool for screening sleep disorders is becoming more urgent. This study developed a computerized adaptive testing for sleep disorders (CAT-SD). Methods: A large sample of 1,304 participants was recruited to construct the item pool of CAT-SD and to investigate the psychometric characteristics of CAT-SD. More specifically, firstly the analyses of unidimensionality, model fit, item fit, item discrimination parameter and differential item functioning (DIF) were conducted to construct a final item pool which meets the requirements of item response theory (IRT) measurement. In addition, a simulated CAT study with real response data of participants was performed to investigate the psychometric characteristics of CAT-SD, including reliability, validity and predictive utility (sensitivity and specificity). Results: The final unidimensional item bank of the CAT-SD not only had good item fit, high discrimination and no DIF; Moreover, it had acceptable reliability, validity and predictive utility. Conclusions: The CAT-SD could be used as an effective and accurate assessment tool for measuring individuals' severity of the sleep disorders and offers a bran-new perspective for screening of sleep disorders with psychological scales.

scholarly journals Calibration of the food parenting practice (FPP) item bank: tools for improving the measurement of food parenting practices of parents of 5–12-year-old children

2020 ◽  
Vol 17 (1) ◽  
Author(s):  
Louise C. Mâsse ◽  
Teresia M. O’Connor ◽  
Yingyi Lin ◽  
Sheryl O. Hughes ◽  
Claire N. Tugault-Lafleur ◽  
...  
Keyword(s):  
Conceptual Framework ◽  
Parenting Practices ◽  
Computerized Adaptive Testing ◽  
Short Form ◽  
Adaptive Testing ◽  
Item Bank ◽  
Short Version ◽  
Parenting Practice ◽  
Practice Item ◽  
Improve Measurement

Abstract Purpose There has been a call to improve measurement rigour and standardization of food parenting practices measures, as well as aligning the measurement of food parenting practices with the parenting literature. Drawing from an expert-informed conceptual framework assessing three key domains of food parenting practices (autonomy promotion, control, and structure), this study combined factor analytic methods with Item Response Modeling (IRM) methodology to psychometrically validate responses to the Food Parenting Practice item bank. Methods A sample of 799 Canadian parents of 5–12-year-old children completed the Food Parenting Practice item bank (129 items measuring 17 constructs). The factorial structure of the responses to the item bank was assessed with confirmatory factor analysis (CFA), confirmatory bi-factor item analysis, and IRM. Following these analyses, differential Item Functioning (DIF) and Differential Response Functioning (DRF) analyses were then used to test invariance properties by parents’ sex, income and ethnicity. Finally, the efficiency of the item bank was examined using computerized adaptive testing simulations to identify the items to include in a short form. Results Overall, the expert-informed conceptual framework was predominantly supported by the CFA as it retained the same 17 constructs included in the conceptual framework with the exception of the access/availability and permissive constructs which were respectively renamed covert control and accommodating the child to better reflect the content of the final solution. The bi-factor item analyses and IRM analyses revealed that the solution could be simplified to 11 unidimensional constructs and the full item bank included 86-items (empirical reliability from 0.78 to 0.96, except for 1 construct) and the short form had 48 items. Conclusion Overall the food parenting practice item bank has excellent psychometric properties. The item bank includes an expanded version and short version to meet various study needs. This study provides more efficient tools for assessing how food parenting practices influence child dietary behaviours. Next steps are to use the IRM calibrated item bank and draw on computerized adaptive testing methodology to administer the item bank and provide flexibility in item selection.

Development of a Computerized Adaptive Test for Assessing Activities of Daily Living in Outpatients With Stroke

2013 ◽  
Vol 93 (5) ◽  
pp. 681-693 ◽  
Author(s):  
I-Ping Hsueh ◽  
Jyun-Hong Chen ◽  
Chun-Hou Wang ◽  
Wen-Hsuan Hou ◽  
Ching-Lin Hsieh
Keyword(s):  
Activities Of Daily Living ◽  
Daily Living ◽  
Computerized Adaptive Testing ◽  
Outcome Measurement ◽  
Theory Model ◽  
Item Bank ◽  
Stopping Rules ◽  
Computerized Adaptive Test ◽  
Maximum Test ◽  
Time Required

Background An efficient, reliable, and valid measure for assessing activities of daily living (ADL) function is useful to improve the efficiency of patient management and outcome measurement. Objective The purpose of this study was to construct a computerized adaptive testing (CAT) system for measuring ADL function in outpatients with stroke. Design Two cohort studies were conducted at 6 hospitals in Taiwan. Methods A candidate item bank (44 items) was developed, and 643 outpatients were interviewed. An item response theory model was fitted to the data and estimated the item parameters (eg, difficulty and discrimination) for developing the ADL CAT. Another sample of 51 outpatients was interviewed to examine the concurrent validity and efficiency of the CAT. The ADL CAT, as the outcome measure, and the Barthel index (BI) and Frenchay Activities index (FAI) were administered on the second group of participants. Results Ten items did not satisfy the model's expectations and were deleted. Thirty-four items were included in the final item bank. Two stopping rules (ie, reliability coefficient >.9 and maximum test length of 7 items) were set for the CAT. The participants' ADL scores had an average reliability of .93. The CAT scores were highly associated with those of the full 34 items (Pearson r=.98). The scores of the CAT were closely correlated with those of the combined BI and FAI (r=.82). The time required to complete the CAT was about one fifth of the time used to administer both the BI and FAI. Limitations The participants were outpatients living in the community. Further studies are needed to cross-validate the results. Conclusions The results demonstrated that the ADL CAT is quick to administer, reliable, and valid in outpatients with stroke.

Sign in / Sign up

Export Citation Format

Share Document