Internal Consistency and Inter-Rater Reliability of the Questions About Behavioral Function (QABF) Rating Scale When Used by Teachers and Paraprofessionals

2014 ◽  
Vol 37 (2) ◽  
pp. 347-364 ◽  
Author(s):  
Michael E. May ◽  
Yanyan Sheng ◽  
Morgan Chitiyo ◽  
Rachel C. Brandt ◽  
Abigail P. Howe
Keyword(s):  
Internal Consistency ◽  
Rating Scale ◽  
Behavioral Function ◽  
Rater Reliability

Mansoura Fiberoptic Endoscopic Evaluation of Swallowing Residue Rating Scale (MFRRS): An Anatomically Based Tool – A Preliminary Study

2020 ◽  
pp. 1-13
Author(s):  
Aliaa Sabry ◽  
James L. Coyle ◽  
Tamer Abou-Elsaad
Keyword(s):  
Internal Consistency ◽  
Rating Scale ◽  
Pyriform Sinus ◽  
Hard Copy ◽  
Kappa Statistics ◽  
Endoscopic Evaluation ◽  
Rater Reliability ◽  
Functional Problem ◽  
Excellent Internal Consistency ◽  
Residue Levels

<b><i>Objective:</i></b> The aim of this work was to design an anatomically based scale for judging post-swallow residue in the pharyngeal cavities, for use during the fiberoptic endoscopic evaluation of swallowing (FEES) in patients with dysphagia, and to assess its feasibility. <b><i>Methods:</i></b> Two 7-point ordinal scales (one for vallecular residue and one for pyriform sinus residue), were developed using detailed anatomic landmarks to denote residue levels. Hard copy color images of a specified frame, from 210 videos of 30 adult FEES evaluations demonstrating the range of all possible residue patterns, were selected (<i>n</i> = 56 valleculae, 62 pyriform sinuses). Half of these images were used to train 4 raters. The remaining half of the images were randomly ordered and rated by the trained raters. Two weeks later the same images were randomized again, and each rater re-analyzed them. The inter- and intra-rater reliability and criterion validity were determined using the kappa statistics and their standard errors. The internal consistency of the items in MFRRS was examined. <b><i>Results:</i></b> MFRRS showed strong inter-rater reliability (valleculae, κ = 0.832 ± 0.038; pyriform sinus, κ = 0.855 ± 0.034), almost perfect intra-rater reliability (valleculae, κ = 0.964 ± 0.018; pyriform sinus, κ = 0.962 ± 0.02), almost perfect concurrent validity (valleculae, κ = 0.968 ± 0.020; pyriform sinus, κ = 0.0971 ± 0.017), and excellent internal consistency (valleculae, Cronbach’s α = 0.990; pyriform sinus, Cronbach’s α = 0.985). <b><i>Conclusion:</i></b> MFRRS is a feasible and reliable, anatomically based tool that can provide more accurate pharyngeal residue judgments. The optimized description of residue accumulation patterns can contribute to a better overall description of the functional problem and future description of dysphagia phenotypes.

The behavior flexibility rating scale-revised (BFRS-R): Factor analysis, internal consistency, inter-rater and intra-rater reliability, and convergent validity

2008 ◽  
Vol 29 (5) ◽  
pp. 398-407 ◽  
Author(s):  
Nienke Peters-Scheffer ◽  
Robert Didden ◽  
Vanessa A. Green ◽  
Jeff Sigafoos ◽  
Hubert Korzilius ◽  
...  
Keyword(s):  
Factor Analysis ◽  
Internal Consistency ◽  
Convergent Validity ◽  
Rating Scale ◽  
Rater Reliability ◽  
R Factor

scholarly journals The Development and Psychometric Evaluation of the Group Schema Therapy Rating Scale – Revised

2018 ◽  
Vol 46 (5) ◽  
pp. 601-618
Author(s):  
Emily Bastick ◽  
Suili Bot ◽  
Simone J. W. Verhagen ◽  
Gerhard Zarbock ◽  
Joan Farrell ◽  
...  
Keyword(s):  
Internal Consistency ◽  
Discriminant Validity ◽  
Rating Scale ◽  
Psychometric Evaluation ◽  
Schema Therapy ◽  
Group Cohesiveness ◽  
Rater Reliability ◽  
Group Format ◽  
Training And Supervision ◽  
Fidelity Measure

Background: Recent research has supported the efficacy of schema therapy as a treatment for personality disorders. A group format has been developed (group schema therapy; GST), which has been suggested to improve both the clinical and cost-effectiveness of the treatment. Aims: Efficacy studies of GST need to assess treatment fidelity. The aims of the present study were to improve, describe and evaluate a fidelity measure for GST, the Group Schema Therapy Rating Scale – Revised (GSTRS-R). Method: Following a pilot study on an initial version of the scale (GSTRS), items were revised and guidelines were modified in order to improve the reliability of the scale. Students highly experienced with the scale rated recorded GST therapy sessions using the GSTRS-R in addition to a group cohesion measure, the Harvard Community Health Plan Group Cohesiveness Scale – II (GCS-II). The scores were used to assess internal consistency and inter-rater reliability. Discriminant validity was assessed by comparing the scores on the GSTRS-R with the GCS-II. Results: The GSTRS-R displayed substantial internal consistency and inter-rater reliability, and adequate discriminate validity, evidenced by a weak positive correlation with the GCS-II. Conclusions: Overall, the GSTRS-R is a reliable tool that may be useful for evaluating therapist fidelity to GST model, and assisting GST training and supervision. Initial validity was supported by a weak association with GCS-II, indicating that although associated with cohesiveness, the instrument also assesses factors specific to GST. Limitations are discussed.

51. Structured assessment format for evaluating operative reports (SAFE-OR) in general surgery

2007 ◽  
Vol 30 (4) ◽  
pp. 55
Author(s):  
A. Vergis ◽  
L. Gillman ◽  
M. Taylor ◽  
S. Minor ◽  
J. Park
Keyword(s):  
Construct Validity ◽  
Internal Consistency ◽  
General Surgery ◽  
Rating Scale ◽  
Assessment Instrument ◽  
Sigmoid Colectomy ◽  
Rater Reliability ◽  
Global Quality ◽  
Canadian Association ◽  
Assessment Format

This study determined the construct validity, inter-rater reliability and internal consistency of a “Structured Assessment Format for Evaluating Operative Reports” (SAFE-OR) in general surgery. The assessment instrument was developed using consensus criteria set forth by the Canadian Association of General Surgeons. It includes a structured assessment and a global quality rating scale. Residents divided into novice and experienced groups viewed and dictated a video-taped laparoscopic sigmoid colectomy. Transcriptions were then graded by blinded, independent faculty evaluators using SAFE-OR. Twenty-one residents participated in the study. Mean structured assessment scores (out of 44) were significantly lower for novice versus experienced residents (23.3 ± 5.2 vs 34.1 ± 6.0, t=0.001). Mean global quality scores (out of 45) were similarly lower for novice residents (25.6 ± 4.7 vs 35.9 ± 7.6, t=0.006). Inter-class correlation coefficients were 0.98 (95% CI 0.96-0.99) for structured assessment and 0.93 (95% CI 0.83-0.97) for global quality scales. Cronbach’s alpha coefficients for internal consistency were 0.85 for structured assessment and 0.96 for global quality assessment scales. SAFE-OR demonstrates significant construct validity, excellent inter-rater reliability and high internal consistency. This tool will allow educators to objectively evaluate the quality of trainee operative reports and ultimately provide a mechanism for implementing, monitoring, and refining curriculum for operative dictation communication skills. Moore R. The dictated operative note: important but is it being taught? Journal of the American College of Surgeons 2000; 190(5):639-40. Novitsky Y, Sing R, et al. Prospective, blinded evaluation of accuracy of operative reports dictated by surgical residents. The American Surgeon 2005; 71(8):627-31. Wanzel K, Ward R, et al. Teaching the surgical craft: From selection to certification. Current Problems in Surgery 2002; 39(6):573-659.

scholarly journals Development of the Therapist Empathy Scale

2013 ◽  
Vol 42 (3) ◽  
pp. 339-354 ◽  
Author(s):  
Suzanne E. Decker ◽  
Charla Nich ◽  
Kathleen M. Carroll ◽  
Steve Martino
Keyword(s):  
Internal Consistency ◽  
Convergent Validity ◽  
Discriminant Validity ◽  
Rating Scale ◽  
Single Factor ◽  
Client Outcome ◽  
Rater Reliability ◽  
Empathy Scale ◽  
Therapist Empathy ◽  
The Impact

Background: Few measures exist to examine therapist empathy as it occurs in session. Aims: A 9-item observer rating scale, called the Therapist Empathy Scale (TES), was developed based on Watson's (1999) work to assess affective, cognitive, attitudinal, and attunement aspects of therapist empathy. The aim of this study was to evaluate the inter-rater reliability, internal consistency, and construct and criterion validity of the TES. Method: Raters evaluated therapist empathy in 315 client sessions conducted by 91 therapists, using data from a multi-site therapist training trial (Martino et al., 2010) in Motivational Interviewing (MI). Results: Inter-rater reliability (ICC = .87 to .91) and internal consistency (Cronbach's alpha = .94) were high. Confirmatory factor analyses indicated some support for single-factor fit. Convergent validity was supported by correlations between TES scores and MI fundamental adherence (r range .50 to .67) and competence scores (r range .56 to .69). Discriminant validity was indicated by negative or nonsignificant correlations between TES and MI-inconsistent behavior (r range .05 to −.33). Conclusions: The TES demonstrates excellent inter-rater reliability and internal consistency. Results indicate some support for a single-factor solution and convergent and discriminant validity. Future studies should examine the use of the TES to evaluate therapist empathy in different psychotherapy approaches and to determine the impact of therapist empathy on client outcome.

scholarly journals Validation of the Italian version of the Functioning Assessment Short Test (FAST) for bipolar disorder

2012 ◽  
Vol 22 (2) ◽  
pp. 187-194 ◽  
Author(s):  
A. Barbato ◽  
L. Bossini ◽  
S. Calugi ◽  
B. D'Avanzo ◽  
A. Fagiolini ◽  
...  
Keyword(s):  
Bipolar Disorder ◽  
Psychometric Properties ◽  
Internal Consistency ◽  
Discriminant Validity ◽  
Rating Scale ◽  
Italian Population ◽  
Italian Version ◽  
Rater Reliability ◽  
Short Test ◽  
Functioning Assessment

Aim.The Functioning Assessment Short Test (FAST) is a useful instrument for the assessment of overall functioning of people with bipolar disorder, showing good psychometric properties. The aim of this study is to validate the Italian version of FAST.Methods.Translation and back-translation of the original FAST Spanish version were performed. Participants with bipolar disorder (n = 132) and healthy controls (n = 132) completed the FAST as a part of an assessment package including the Montgomery–Asberg Depression Rating Scale and the Young Mania Rating Scale. Internal consistency, inter-rater reliability, construct and discriminant validity were assessed.Results.The FAST Italian version showed good internal consistency, inter-rater reliability and discriminant validity. The cut-off discriminating patients from controls was 15, with a sensitivity of 0.79 and a specificity of 0.80. Principal component analysis with oblique rotation showed factor loadings consistent with the a priori structure of the instrument.Conclusions.This study confirmed the psychometric properties of FAST and extended its generalization and validity to the Italian population.

Development and Validation of an Instrument to Assess Horseback Mobility in Hippotherapy

2021 ◽  
pp. 003151252110365
Author(s):  
Alessandra V. Prieto ◽  
Kênnea Martins Almeida Ayupe ◽  
Ana C. A. Abreu ◽  
Paulo J. B. Gutierres Filho
Keyword(s):  
Correlation Coefficient ◽  
Factor Structure ◽  
Internal Consistency ◽  
Rater Reliability ◽  
Intra Class Correlation ◽  
Exploratory Factor Analyses ◽  
Excellent Internal Consistency ◽  
Development And Validation ◽  
Functional Gain ◽  
Intra Class Correlation Coefficient

Improvement in rider mobility represents an important functional gain for people with disabilities undergoing hippotherapy. However, there is no validated measuring instrument to track and document the rider's progress in riding activities. In this study, we aimed to develop and establish validity evidence for an instrument to assess hippotherapy participants’ mobility on horseback. We report on this development through the stages of: (a) content validation, (b) construct validation, (c) inter- and intra-rater reliability and (d) internal consistency analysis. We evaluated its factor structure with exploratory factor analyses, calculated values for inter- and intra-rater reliability using the intra-class correlation coefficient, and calculated its internal consistency using Cronbach's alpha. We followed recommendations by the Guidelines for Reporting Reliability and Agreement Studies. We found good inter-rater reliability (intra-class correlation coefficient – ICC = 0.991–0.999) and good intra-rater reliability (ICC = 0.997–1.0), and there was excellent internal consistency (Cronbach's α = 0.937–0.999). The instrument’s factor structure grouped its three domains into one factor. As this instrument is theoretically consistent and has been found to be appropriate and reliable for its intended use, it is now available for the measurement of horseback mobility among hippotherapy riders.

scholarly journals Deterioration of housing environment and mental health of asylum seekers – a multi-level analysis

2019 ◽  
Vol 29 (Supplement_4) ◽  
Author(s):  
A Mohsenpour ◽  
L Biddle ◽  
K Bozorgmehr
Keyword(s):  
Mental Health ◽  
Internal Consistency ◽  
Odds Ratio ◽  
Asylum Seekers ◽  
Rapid Assessment ◽  
Living Environment ◽  
Generalized Anxiety ◽  
Rater Reliability ◽  
General Anxiety ◽  
Housing Environment

Abstract Background Asylum seekers arriving in Germany are assigned to a local district and an accommodation centre wherein. We developed and validated a six-item questionnaire for rapid assessment of housing deterioration and investigated its association with inhabitant mental health. Methods Using cross-sectional data from a state-wide survey in Germany, we applied a random-effects modelling approach to estimate the exposure effect of housing deterioration on depression and general anxiety among a random sample of asylum seekers, using validated instruments (GAD2/PHQ2) for outcome variables. Housing deterioration was assessed on six items (windows/glass, walls/roof, garbage, graffiti, outside spaces, overall living environment) resulting in a deterioration score as exposure of interest. Additionally, we assessed the instrument’s intra- and inter-rater reliability and internal consistency. Results Of the 412 asylum seekers living in 58 accommodation centres, 45.7% reported symptoms of depression and 45.0% suffered general anxiety. Most centres (76.7%) were based in urban municipalities and 45.7% of inhabitants were living in an accommodation centre hosting ≥51 inhabitants. Preliminary adjusted odds ratio for accommodation centres with highest deterioration (Q 4) was 2.07 (0.67-6.40) for generalized anxiety, compared to 1.17 (0.45-3.08) for centres with lowest deterioration (Q 1). For depression, preliminary odds ratio was 1.92 (0.87-4.27) compared to 1.26 (0.63-2.50). The validation study confirmed inter-/intra-rater reliability (Brennan-Prediger coefficient: 0.81 and 0.92, respectively) and internal consistency (Crohnbach’s α: 0.80). Conclusions There are higher odds ratios for generalized anxiety disorder and depression among asylum seekers based on higher deterioration of housing environment. A questionnaire for rapid deterioration assessment and identification of accommodations needing further evaluation has been developed and successfully validated. Key messages Deterioration of small-scale housing environment is associated with poorer mental health for asylum seekers living in accommodation centres. A highly reliable new tool has been developed for rapid assessment of deterioration status of accommodation centres and identification of those needing further evaluation.

Bringing the Automation-Related Complacency Scale into the 21st Century

2020 ◽  
Vol 64 (1) ◽  
pp. 1228-1232
Keyword(s):  
Internal Consistency ◽  
21St Century ◽  
Rating Scale ◽  
System Failure ◽  
Individual Values ◽  
Cronbach's Alpha ◽  
Important Measure ◽  
Frequency Of Use ◽  
Performance Error ◽  
The Individual

Complacency potential is an important measure to avoid performance error, such as neglecting to detect a system failure. This study updates and expands upon Singh, Molloy, and Parasuraman’s 1993 Complacency-Potential Rating Scale (CPRS). We updated and expanded the CPRS questions to include technology commonly used today and how frequently the technology is used. The goal of our study was to update the scale, analyze for factor shifts and internal consistency, and to explore correlations between the individual values for each factor and the frequency of use questions. We hypothesized that the factors would not shift from the original and the revised CPRS’s four subscales. Our research found that the revised CPRS consisted of only three subscales with the following Cronbach’s Alpha values: Confidence: 0.599, Safety/Reliability: 0.534, and Trust: 0.201. Correlations between the subscales and the revised complacency-potential and the frequency of use questions are also discussed.

scholarly journals Psychometric Evaluation of the Persian Version of Barkley Adult Attention Deficit/Hyperactivity Disorder Screening Tool among the Elderly

Scientifica ◽  
2017 ◽  
Vol 2017 ◽  
pp. 1-6 ◽  
Author(s):  
Mostafa Sadeghi ◽  
Homayoun Sadeghi-Bazargani ◽  
Shahrokh Amiri
Keyword(s):  
Attention Deficit Hyperactivity Disorder ◽  
Internal Consistency ◽  
Attention Deficit ◽  
Concurrent Validity ◽  
Rating Scale ◽  
Retest Reliability ◽  
Hyperactivity Disorder ◽  
Persian Version ◽  
Test Retest Reliability ◽  
Tabriz City

Background. The Barkley Adult Attention Deficit/Hyperactivity Disorder (ADHD) Rating Scale-IV (BAARS-IV) was developed, and it demonstrated good psychometric properties. The BAARS-IV includes 27 questions on the symptoms of adult ADHD. The purpose of the present study is to investigate the psychometric testing of the Persian version of BAARS-IV among the elderlies in Tabriz City. Method. This cross-sectional study was conducted in Tabriz City—in the west of Iran—in 2015 via enrolling of 121 old-aged people. We did the process of translation and adaptation of BAARS-IV and examined its concurrent validity, internal consistency, and test-retest reliability. Result. The BAARS-IV demonstrated good internal consistency and test-retest reliability. Correlations between the BAARS-IV and the CAARS-S: SV were high and evidence supporting concurrent validity was revealed. Cronbach’s alpha for the overall scale and subscales stood at 0.89, 0.81, 0.66, 0.56, and 0.82, respectively. Conclusion. The Persian BAARS-IV showed acceptable reliability and validity. BAARS-IV was determined to be composed of internally consistent and psychometrically sound items.

Sign in / Sign up

Export Citation Format

Share Document