scholarly journals Examining Interrater Reliability and Validity of a Paediatric Cardiopulmonary Physiotherapy Discharge Tool

2014 ◽  
Vol 66 (2) ◽  
pp. 153-159 ◽  
Author(s):  
Jamil Lati ◽  
Vanessa Pellow ◽  
Jeannine Sproule ◽  
Dina Brooks ◽  
Cindy Ellerton
Keyword(s):  
Interrater Reliability ◽  
Reliability And Validity

Development and Initial Validation of a Project-Based Rubric to Assess the Systems-Based Practice Competency of Residents in the Clinical Chemistry Rotation of a Pathology Residency

2014 ◽  
Vol 138 (6) ◽  
pp. 809-813
Author(s):  
Carolyn R. Vitek ◽  
Jane C. Dale ◽  
Henry A. Homburger ◽  
Sandra C. Bryant ◽  
Amy K. Saenger ◽  
...  
Keyword(s):  
Critical Thinking ◽  
Interrater Reliability ◽  
Clinical Chemistry ◽  
Core Competencies ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Thinking Skills ◽  
Project Evaluation ◽  
Critical Thinking Skills

Context.— Systems-based practice (SBP) is 1 of 6 core competencies required in all resident training programs accredited by the Accreditation Council for Graduate Medical Education. Reliable methods of assessing resident competency in SBP have not been described in the medical literature. Objective.— To develop and validate an analytic grading rubric to assess pathology residents' analyses of SBP problems in clinical chemistry. Design.— Residents were assigned an SBP project based upon unmet clinical needs in the clinical chemistry laboratories. Using an iterative method, we created an analytic grading rubric based on critical thinking principles. Four faculty raters used the SBP project evaluation rubric to independently grade 11 residents' projects during their clinical chemistry rotations. Interrater reliability and Cronbach α were calculated to determine the reliability and validity of the rubric. Project mean scores and range were also assessed to determine whether the rubric differentiated resident critical thinking skills related to the SBP projects. Results.— Overall project scores ranged from 6.56 to 16.50 out of a possible 20 points. Cronbach α ranged from 0.91 to 0.96, indicating that the 4 rubric categories were internally consistent without significant overlap. Intraclass correlation coefficients ranged from 0.63 to 0.81, indicating moderate to strong interrater reliability. Conclusions.— We report development and statistical analysis of a novel SBP project evaluation rubric. The results indicate the rubric can be used to reliably assess pathology residents' critical thinking skills in SBP.

Reliability and Validity Analyses of the Youth Level of Service/Case Management Inventory

2005 ◽  
Vol 32 (3) ◽  
pp. 329-344 ◽  
Author(s):  
Fred Schmidt ◽  
Robert D. Hoge ◽  
Lezlie Gomes
Keyword(s):  
Juvenile Offenders ◽  
Case Management ◽  
Interrater Reliability ◽  
Assessment Tool ◽  
Reliability And Validity ◽  
Level Of Service ◽  
Risk Level ◽  
Criminogenic Needs ◽  
Health Assessments ◽  
Males And Females

The Youth Level of Service/Case Management Inventory (YLS/CMI) is a structured assessment tool designed to facilitate the effective intervention and rehabilitation of juvenile offenders by assessing each youth’s risk level and criminogenic needs. The present study examined the YLS/CMI’s reliability and validity in a sample of 107 juvenile offenders who were court-referred for mental health assessments. Results demonstrated the YLS/CMI’s internal consistency and interrater reliability. Moreover, the instrument’s predictive validity was substantiated on a number of recidivism measures for both males and females. Limitations of the current findings are discussed.

“You Say Severe, I Say Mild”: Toward an Empirical Classification of Dysarthria Severity

2021 ◽  
pp. 1-18
Author(s):  
Kaila L. Stipancic ◽  
Kira M. Palmer ◽  
Hannah P. Rowe ◽  
Yana Yunusova ◽  
James D. Berry ◽  
...  
Keyword(s):  
Speech Intelligibility ◽  
Interrater Reliability ◽  
Reliability And Validity ◽  
Roc Curves ◽  
Speaking Rate ◽  
Response Options ◽  
Cutoff Points ◽  
Healthy Control ◽  
Dysarthric Speech ◽  
Surrogate Measures

Purpose: The main purpose of this study was to create an empirical classification system for speech severity in patients with dysarthria secondary to amyotrophic lateral sclerosis (ALS) by exploring the reliability and validity of speech-language pathologists' (SLPs') ratings of dysarthric speech. Method: Ten SLPs listened to speech samples from 52 speakers with ALS and 20 healthy control speakers. SLPs were asked to rate the speech severity of the speakers using five response options: normal, mild, moderate, severe, and profound. Four severity-surrogate measures were also calculated: SLPs transcribed the speech samples for the calculation of speech intelligibility and rated the effort it took to understand the speakers on a visual analog scale. In addition, speaking rate and intelligible speaking rate were calculated for each speaker. Intrarater and interrater reliability were calculated for each measure. We explored the validity of clinician-based severity ratings by comparing them to the severity-surrogate measures. Receiver operating characteristic (ROC) curves were conducted to create optimal cutoff points for defining dysarthria severity categories. Results: Intrarater and interrater reliability for the clinician-based severity ratings were excellent and were comparable to reliability for the severity-surrogate measures explored. Clinician severity ratings were strongly associated with all severity-surrogate measures, suggesting strong construct validity. We also provided a range of values for each severity-surrogate measure within each severity category based on the cutoff points obtained from the ROC analyses. Conclusions: Clinician severity ratings of dysarthric speech are reliable and valid. We discuss the underlying challenges that arise when selecting a stratification measure and offer recommendations for a classification scheme when stratifying patients and research participants into speech severity categories.

scholarly journals Teleassessment of Gait and Gait Aids: Validity and Interrater Reliability

2020 ◽  
Vol 100 (4) ◽  
pp. 708-717
Author(s):  
Kavita Venkataraman ◽  
Kristopher Amis ◽  
Lawrence R Landerman ◽  
Kevin Caves ◽  
Gerald C Koh ◽  
...  
Keyword(s):  
Repeated Measures ◽  
Interrater Reliability ◽  
Video Quality ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Slow Motion ◽  
Frame Rate ◽  
Normal Speed ◽  
Lower Frame ◽  
Rehabilitation Needs

Abstract Background Gait and mobility aid assessments are important components of rehabilitation. Given the increasing use of telehealth to meet rehabilitation needs, it is important to examine the feasibility of such assessments within the constraints of telerehabilitation. Objective The objective of this study was to examine the reliability and validity of the Tinetti Performance-Oriented Mobility Assessment gait scale (POMA-G) and cane height assessment under various video and transmission settings to demonstrate the feasibility of teleassessment. Design This repeated-measures study compared the test performances of in-person, slow motion (SM) review, and normal-speed (NS) video ratings at various fixed frame rates (8, 15, and 30 frames per second) and bandwidth (128, 384, and 768 kB/s) configurations. Methods Overall bias, validity, and interrater reliability were assessed for in-person, SM video, and NS video ratings, with SM video rating as the gold standard, as well as for different frame rate and bandwidth configurations within NS videos. Results There was moderate to good interrater reliability for the POMA-G (intraclass correlation coefficient [ICC] = 0.66–0.77 across all configurations) and moderate validity for in-person (β = 0.62; 95% confidence interval [CI] = 0.37–0.87) and NS video (β = 0.74; 95% CI = 0.67–0.80) ratings compared with the SM video rating. For cane height, interrater reliability was good (ICC = 0.66–0.77), although it was significantly lower at the lowest frame rate (8 frames per second) (ICC = 0.66; 95% CI = 0.54–0.76) and bandwidth (128 kB/s) (ICC = 0.69; 95% CI = 0.57–0.78) configurations. Validity for cane height was good for both in-person (β = 0.80; 95% CI = 0.62–0.98) and NS video (β = 0.86; 95% CI = 0.81–0.90) ratings compared with SM video rating. Limitations Some lower frame rate and bandwidth configurations may limit the reliability of remote cane height assessments. Conclusions Teleassessment for POMA-G and cane height using typically available internet and video quality is feasible, valid, and reliable.

scholarly journals Alberta Infant Motor Scale: Reliability and Validity When Used on Preterm Infants in Taiwan

2000 ◽  
Vol 80 (2) ◽  
pp. 168-178 ◽  
Author(s):  
Suh-Fang Jeng ◽  
Kuo-Inn Tsou Yau ◽  
Li-Chiou Chen ◽  
Shu-Fang Hsiao
Keyword(s):  
Preterm Infants ◽  
Interrater Reliability ◽  
Physical Therapist ◽  
Intraclass Correlation ◽  
Reliability And Validity ◽  
Correlation Coefficients ◽  
Intraclass Correlation Coefficients ◽  
Scale Reliability ◽  
Scale Scores ◽  
Acceptable Reliability

Abstract Background and Purpose. The goal of this study was to examine the reliability and validity of measurements obtained with the Alberta Infant Motor Scale (AIMS) for evaluation of preterm infants in Taiwan. Subjects. Two independent groups of preterm infants were used to investigate the reliability (n=45) and validity (n=41) for the AIMS. Methods. In the reliability study, the AIMS was administered to the infants by a physical therapist, and infant performance was videotaped. The performance was then rescored by the same therapist and by 2 other therapists to examine the intrarater and interrater reliability. In the validity study, the AIMS and the Bayley Motor Scale were administered to the infants at 6 and 12 months of age to examine criterion-related validity. Results. Intraclass correlation coefficients (ICCs) for intrarater and interrater reliability of measurements obtained with the AIMS were high (ICC=.97–.99). The AIMS scores correlated with the Bayley Motor Scale scores at 6 and 12 months (r=.78 and .90), although the AIMS scores at 6 months were only moderately predictive of the motor function at 12 months (r=.56). Conclusion and Discussion. The results suggest that measurements obtained with the AIMS have acceptable reliability and concurrent validity but limited predictive value for evaluating preterm Taiwanese infants.

Reliability and Validity of the Hierarchic Dementia Scale

1994 ◽  
Vol 6 (1) ◽  
pp. 87-94 ◽  
Author(s):  
Lisa Rönnberg ◽  
Kjerstin Ericsson
Keyword(s):  
Cognitive Deficits ◽  
Concurrent Validity ◽  
Interrater Reliability ◽  
Mixed Type ◽  
Reliability And Validity ◽  
Kappa Coefficient ◽  
Severe Dementia ◽  
Valid Instrument ◽  
Test Retest Reliability

The aim of the study was to apply a method for measuring cognitive functioning in severely demented patients. Reliability and validity of the Hierarchic Dementia Scale (HDS) were tested. Fifty patients with dementia of the Alzheimer's type (DAT), multi-infarct dementia (MID), and dementia of mixed type (MIX) were studied. The interrater reliability was satisfactory as estimated by means of a kappa coefficient. The test retest reliability was rs = .96. The concurrent validity of the HDS, as measured by the correlation between the HDS and the MMSE, was rs = .86, and between the HDS and the CDR was rs = −.71. The results indicate that HDS is a useful and valid instrument for determination of the heterogeneous cognitive deficits in severe dementia.

Q by Q Interview as a Predictor of Success in Second Language Learning

1971 ◽  
Vol 29 (1) ◽  
pp. 331-337 ◽  
Author(s):  
James J. Asher
Keyword(s):  
Language Learning ◽  
Second Language Learning ◽  
Interrater Reliability ◽  
Reliability And Validity ◽  
Language Aptitude ◽  
Range Restriction ◽  
Aptitude Tests ◽  
Intensive Training ◽  
Selection Interview ◽  
Predictor Of Success

This study was designed to find the reliability and validity of the selection interview used to predict student success after 6 wk. of intensive training in Arabic, German, Hungarian, Polish, Russian, Turkish, and Vietnamese. The criterion measures were grades and instructor ratings for listening, speaking, reading, and writing. The results indicated that interview reliability and validity were a function of range restriction. Given minimal range restriction, interrater reliability was extremely high and validity was substantial. Intelligence and language aptitude tests showed even higher validity than the interview.

The Reliability and Validity of the Health of the Nation Outcome Scales: Validation in Relation to Patient Derived Measures

2000 ◽  
Vol 34 (3) ◽  
pp. 504-511 ◽  
Author(s):  
Robert Brooks
Keyword(s):  
Mental Health ◽  
Psychometric Properties ◽  
Interrater Reliability ◽  
New South ◽  
Short Form ◽  
Reliability And Validity ◽  
The Third ◽  
South Wales ◽  
Sf 36 ◽  
Before And After

Objective: The Health of Nation Outcome Scales (HoNOS) was developed to assess mental health outcomes. The aim of the studies is to examine the psychometric properties, reliability and validity of the HoNOS. Method: Three studies were conducted within St John of God Hospitals in New South Wales, Australia. They examined the reliability and the validity of the HoNOS. The first study examined the interrater reliability of the HoNOS, before and after staff training in the use of the HoNOS. The second study examined the validity of the HoNOS with the Symptom Checklist 90 Revised (SCL90-R) and the third study examined the validity of the HoNOS with the Short-Form 36 (SF-36). Results: The first study showed an improvement in the interrater reliability (IRR) of the HoNOS due to training. However, a generally unsatisfactory IRR (range 0.50–0.65) was achieved. The second study found no correlation between the SCL90-R and the HoNOS on admission (r = 0.04) and discharge (r = 0.06). The third study found no significant correlation between the Mental Component Score of the SF-36 and the HoNOS on admission (r = −0.033) nor on discharge (r = −0.104). Conclusions: The HoNOS has at best moderate interrater reliabilities. Further, the validity of the HoNOS is under question, that is, it does not correlate with a major measure of mental health symptoms, nor with a major measure of health status. As such, it is concluded that the psychometric properties of the HoNOS do not warrant its use as a routine measure.

scholarly journals Reliability and Validity of a Novel Futsal Special Performance Test Designed to Measure Skills and Anaerobic Performance

2019 ◽  
Vol 14 (8) ◽  
pp. 1096-1102 ◽  
Author(s):  
Farid Farhani ◽  
Hamid Rajabi ◽  
Raoof Negaresh ◽  
Ajmol Ali ◽  
Sadegh Amani Shalamzari ◽  
...  
Keyword(s):  
Content Validity ◽  
Interrater Reliability ◽  
Performance Test ◽  
Anaerobic Power ◽  
Reliability And Validity ◽  
Validity And Reliability ◽  
Performance Time ◽  
And Performance ◽  
Different Levels ◽  
Familiarization Session

Purpose: To examine the validity and reliability of a novel futsal special performance test (FSPT) as a measure of futsal performance and skills. Methods: Thirty-six futsal players with different levels of experience were recruited and divided into 2 groups (elite and nonelite). Players participated in 4 sessions (at least 7 d apart): (1) familiarization session, (2) anaerobic power (Wingate test), (3) FSPT trial 1, and (4) FSPT trial 2. The FSPT was carried out on a futsal court (wooden sprung floor) and skills such as dribbling, rotation, long and short passing, and shooting were examined. Content validity was assessed by 6 experienced futsal coaches and instructors. Results: There was a significant correlation between FSPT and various aspects of anaerobic power (r = .5–.91; P ≤ .001). Moreover, significant large correlations were observed between test and retest of FSPT (r = .77; 95% confidence interval [CI], .56–.98; P ≤ .001). All instructors and coaches confirmed the content validity. There was high interrater reliability of the FSPT (r = .89; 95% CI, .85–.93; P < .001). FSPT total time (P = .001), penalty time (P = .022), and performance time (P = .001) were superior in elite relative to nonelite players. Anaerobic power was greater in elite players (P < .001). Conclusion: The results support the use of the FSPT to assess futsal players’ performance in conjunction with skill and anaerobic fitness.

Sign in / Sign up

Export Citation Format

Share Document