scholarly journals Development and Validation of a Comprehensive Genomics Knowledge Scale

2021 ◽  
pp. 1-13
Author(s):  
Michael D. Linderman ◽  
Sabrina A. Suckiel ◽  
Nathan Thompson ◽  
David J. Weiss ◽  
J. Scott Roberts ◽  
...  
Keyword(s):  
Computerized Adaptive Testing ◽  
Classical Test Theory ◽  
Healthcare Providers ◽  
Educational Interventions ◽  
Test Theory ◽  
Dominant Factor ◽  
Genomic Testing ◽  
Genomic Technologies ◽  
Knowledge Scale ◽  
Genomic Literacy

<b><i>Background:</i></b> Genomic testing is increasingly employed in clinical, research, educational, and commercial contexts. Genomic literacy is a prerequisite for the effective application of genomic testing, creating a corresponding need for validated tools to assess genomics knowledge. We sought to develop a reliable measure of genomics knowledge that incorporates modern genomic technologies and is informative for individuals with diverse backgrounds, including those with clinical/life sciences training. <b><i>Methods:</i></b> We developed the GKnowM Genomics Knowledge Scale to assess the knowledge needed to make an informed decision for genomic testing, appropriately apply genomic technologies and participate in civic decision-making. We administered the 30-item draft measure to a calibration cohort (<i>n</i> = 1,234) and subsequent participants to create a combined validation cohort (<i>n</i> = 2,405). We performed a multistage psychometric calibration and validation using classical test theory and item response theory (IRT) and conducted a post-hoc simulation study to evaluate the suitability of a computerized adaptive testing (CAT) implementation. <b><i>Results:</i></b> Based on exploratory factor analysis, we removed 4 of the 30 draft items. The resulting 26-item GKnowM measure has a single dominant factor. The scale internal consistency is α = 0.85, and the IRT 3-PL model demonstrated good overall and item fit. Validity is demonstrated with significant correlation (<i>r =</i> 0.61) with an existing genomics knowledge measure and significantly higher scores for individuals with adequate health literacy and healthcare providers (HCPs), including HCPs who work with genomic testing. The item bank is well suited to CAT, achieving high accuracy (<i>r</i> = 0.97 with the full measure) while administering a mean of 13.5 items. <b><i>Conclusion:</i></b> GKnowM is an updated, broadly relevant, rigorously validated 26-item measure for assessing genomics knowledge that we anticipate will be useful for assessing population genomic literacy and evaluating the effectiveness of genomics educational interventions.

Psychometric Evaluation of a 10-Item Health Insurance Knowledge Scale

2021 ◽  
pp. JNM-D-20-00019
Author(s):  
Tyler G. James ◽  
M. David Miller ◽  
Guy Nicolette ◽  
JeeWon Cheong
Keyword(s):  
College Students ◽  
Health Insurance ◽  
Classical Test Theory ◽  
Psychometric Evaluation ◽  
Test Theory ◽  
Literacy Interventions ◽  
Education Practice ◽  
Health Insurance Literacy ◽  
Psychometric Studies ◽  
Knowledge Scale

BackgroundCollege students are a priority population for health insurance literacy interventions. Yet, there are few psychometric studies on measuring health insurance knowledge – a core construct of health insurance literacy.MethodsWe administered a health insurance survey to 2,250 college students. We applied Classical Test Theory and Item Response Theory methods to estimate psychometric properties of the Kaiser Family Foundation's 10-item health insurance knowledge quiz.ResultsThe scale is unidimensional, and a two-parameter logistic model best fit the data. IRT estimates indicated varying item discriminations (a range: 0.717–2.578) and difficulties (b range: −0.913–1.790). Precision of measurement was maximized for students half a standard deviation below the mean (θ = −0.686) health insurance knowledge ability.ConclusionsThis scale can be used to identify gaps in health insurance knowledge among college students and be applied in clinical and community health education practice.

scholarly journals On the complementarity of classical test theory and item response models: item difficulty estimates and computerized adaptive testing

2015 ◽  
Vol 23 (88) ◽  
pp. 593-610
Author(s):  
Patrícia Costa ◽  
Maria Eugénia Ferrão
Keyword(s):  
Item Response ◽  
Computerized Adaptive Testing ◽  
Item Difficulty ◽  
Classical Test Theory ◽  
Adaptive Testing ◽  
Test Theory ◽  
Partial Credit Model ◽  
Response Models ◽  
Item Response Models ◽  
Classical Test

This study aims to provide statistical evidence of the complementarity between classical test theory and item response models for certain educational assessment purposes. Such complementarity might support, at a reduced cost, future development of innovative procedures for item calibration in adaptive testing. Classical test theory and the generalized partial credit model are applied to tests comprising multiple choice, short answer, completion, and open response items scored partially. Datasets are derived from the tests administered to the Portuguese population of students enrolled in the 4th and 6th grades. The results show a very strong association between the estimates of difficulty obtained from classical test theory and item response models, corroborating the statistical theory of mental testing.

Testing the Assumption of Uncorrelated Errors for Short Scales by Means of Structural Equation Modeling

2014 ◽  
Vol 35 (4) ◽  
pp. 201-211 ◽  
Author(s):  
André Beauducel ◽  
Anja Leue
Keyword(s):  
Structural Equation Modeling ◽  
Negative Correlation ◽  
Structural Equation ◽  
Error Score ◽  
Classical Test Theory ◽  
Test Theory ◽  
Equation Modeling ◽  
Eysenck Personality Questionnaire ◽  
Personality Questionnaire ◽  
True Score

It is shown that a minimal assumption should be added to the assumptions of Classical Test Theory (CTT) in order to have positive inter-item correlations, which are regarded as a basis for the aggregation of items. Moreover, it is shown that the assumption of zero correlations between the error score estimates is substantially violated in the population of individuals when the number of items is small. Instead, a negative correlation between error score estimates occurs. The reason for the negative correlation is that the error score estimates for different items of a scale are based on insufficient true score estimates when the number of items is small. A test of the assumption of uncorrelated error score estimates by means of structural equation modeling (SEM) is proposed that takes this effect into account. The SEM-based procedure is demonstrated by means of empirical examples based on the Edinburgh Handedness Inventory and the Eysenck Personality Questionnaire-Revised.

The Effect of Alternative Scoring Procedures on the Measurement Properties of a Self-Administered Depression Scale

2019 ◽  
Vol 35 (1) ◽  
pp. 55-62 ◽  
Author(s):  
Noboru Iwata ◽  
Akizumi Tsutsumi ◽  
Takafumi Wakita ◽  
Ryuichi Kumagai ◽  
Hiroyuki Noguchi ◽  
...  
Keyword(s):  
Classical Test Theory ◽  
Depression Scale ◽  
Psychometric Testing ◽  
Epidemiologic Studies ◽  
Test Theory ◽  
Measurement Properties ◽  
Irt Model ◽  
Response Alternatives ◽  
Polytomous Item Response ◽  
Θ Point

Abstract. To investigate the effect of response alternatives/scoring procedures on the measurement properties of the Center for Epidemiologic Studies Depression Scale (CES-D) which has the four response alternatives, a polytomous item response theory (IRT) model was applied to the responses of 2,061 workers and university students (1,640 males, 421 females). Test information functions derived from the polytomous IRT analyses on the CES-D data with various scoring procedures indicated that: (1) the CES-D with its standard (0-1-2-3) scoring procedure should be useful for screening to detect subjects with “at high-risk” of depression if the θ point showing the highest information corresponds to the cut-off point, because of its extremely higher information; (2) the CES-D with the 0-1-1-2 scoring procedure could cover wider range of depressive severity, suggesting that this scoring procedure might be useful in cases where more exhaustive discrimination in symptomatology is of interest; and (3) the revised version of CES-D with replacing original positive items into negatively revised items outperformed the original version. These findings have never been demonstrated by the classical test theory analyses, and thus the utility of this kind of psychometric testing should be warranted to further investigation for the standard measures of psychological assessment.

Revision of a Criterion-Referenced Vocabulary Test Using Generalizability Theory

2009 ◽  
Vol 31 (1) ◽  
pp. 81
Author(s):  
Takeaki Kumazawa
Keyword(s):  
Generalizability Theory ◽  
Classical Test Theory ◽  
Performance Testing ◽  
Error Variance ◽  
Test Theory ◽  
Vocabulary Test ◽  
Multiple Sources ◽  
Generalizability Coefficient ◽  
Criterion Referenced ◽  
Generalizability Study

Classical test theory (CTT) has been widely used to estimate the reliability of measurements. Generalizability theory (G theory), an extension of CTT, is a powerful statistical procedure, particularly useful for performance testing, because it enables estimating the percentages of persons variance and multiple sources of error variance. This study focuses on a generalizability study (G study) conducted to investigate such variance components for a paper-pencil multiple-choice vocabulary test used as a diagnostic pretest. Further, a decision study (D study) was conducted to compute the generalizability coefficient (G coefficient) for absolute decisions. The results of the G and D studies indicated that 46% of the total variance was due to the items effect; further, the G coefficient for absolute decisions was low. 古典的テスト理論は尺度の信頼性を測定するため広く用いられている。古典的テスト理論の応用である一般化可能性理論（G理論）は特にパフォーマンステストにおいて有効な分析手法であり、受験者と誤差の要因となる分散成分の割合を測定することができる。本研究では診断テストとして用いられた多岐選択式語彙テストの分散成分を測定するため一般化可能性研究（G研究）を行った。さらに、決定研究（D研究）では絶対評価に用いる一般化可能性係数を算出した。G研究とD研究の結果、項目の分散成分が全体の分散の46%を占め、また信頼度指数は高くなかった。

scholarly journals Exploring the experiences related to postpartum changes: perspectives of mothers and healthcare providers in Iran

2021 ◽  
Vol 21 (1) ◽  
Author(s):  
Mahboobeh Asadi ◽  
Mahnaz Noroozi ◽  
Mousa Alavi
Keyword(s):  
Postpartum Period ◽  
Qualitative Content Analysis ◽  
Healthcare Providers ◽  
Educational Interventions ◽  
Psychological State ◽  
Structured Interviews ◽  
Childbearing Age ◽  
New Period ◽  
Women's Lives ◽  
Field Notes

Abstract Background Numerous changes occur in different aspects of women’s lives in the postpartum period. Women’s adjusting with problems and taking advantage of this opportunity can develop their personality. In this regard, accurate knowledge of their experiences and feelings is necessary to help them to benefit from this period. Therefore, the present study aimed to explore the experiences related to postpartum changes in women. Methods In the present qualitative study, 23 participants, including women of childbearing age who gave birth and healthcare providers (midwives and obstetricians) in Isfahan, Iran were selected using purposive sampling with a maximum variation strategy. Data were collected through in-depth semi structured interviews, field notes, and daily notes, and simultaneously analyzed using the conventional qualitative content analysis. Results The data analysis results led to the extraction of three main categories including “feeling of decreased female attractiveness” (with two sub-categories of “ feeling of decreased beauty” and “feeling of decreased sexual function”), “feeling of insolvency and helplessness” (with two sub-categories of “physical burnout”, and “mental preoccupations”) and “beginning a new period in life” (with three sub-categories of “changing the meaning of life”, “feeling of maturity” and “deepening the communication”). Conclusions Findings of this study can provide a good context for designing interventions to improve the women’s quality of life by explaining and highlighting their experiences in the postpartum period. In this regard, providing sufficient empathy, social and psychological support from family members (especially husband), performing appropriate educational interventions and also regular assessment of women’s psychological state by healthcare providers in postpartum period can reduce their concerns and help to improve their health.

scholarly journals Time Saving Students’ Formative Assessment: Algorithm to Balance Number of Tasks and Result Reliability

2021 ◽  
Vol 11 (13) ◽  
pp. 6048
Author(s):  
Jaroslav Melesko ◽  
Simona Ramanauskaite
Keyword(s):  
Formative Assessment ◽  
Item Response ◽  
Item Difficulty ◽  
Classical Test Theory ◽  
Personalized Learning ◽  
Test Theory ◽  
Time Saving ◽  
Learning Path ◽  
Crucial Component

Feedback is a crucial component of effective, personalized learning, and is usually provided through formative assessment. Introducing formative assessment into a classroom can be challenging because of test creation complexity and the need to provide time for assessment. The newly proposed formative assessment algorithm uses multivariate Elo rating and multi-armed bandit approaches to solve these challenges. In the case study involving 106 students of the Cloud Computing course, the algorithm shows double learning path recommendation precision compared to classical test theory based assessment methods. The algorithm usage approaches item response theory benchmark precision with greatly reduced quiz length without the need for item difficulty calibration.

Sign in / Sign up

Export Citation Format

Share Document