scholarly journals Reevaluating Inter-Rater Reliability in Offender Risk Assessment

2011 ◽  
Author(s):  
Leontien Marjon van der Knaap ◽  
Marise Ph. Born ◽  
L. E. W. Leenarts ◽  
P. Oosterveld
Keyword(s):  
Risk Assessment ◽  
Rater Reliability

Youth Risk Assessment: Inter-Rater Reliability of State Corrections Staff

2012 ◽  
Author(s):  
Patrick J. Kennealy ◽  
Isaias Hernandez ◽  
Jennifer Skeem
Keyword(s):  
Risk Assessment ◽  
Rater Reliability

scholarly journals Inter-Rater Reliability of a Pressure Injury Risk Assessment Scale for Home Care: A Multicenter Cross-Sectional Study

2020 ◽  
Vol Volume 13 ◽  
pp. 2031-2041
Author(s):  
Masushi Kohta ◽  
Takehiko Ohura ◽  
Kunio Tsukada ◽  
Yoshinori Nakamura ◽  
Mishiho Sukegawa ◽  
...  
Keyword(s):  
Risk Assessment ◽  
Home Care ◽  
Injury Risk ◽  
Cross Sectional Study ◽  
Assessment Scale ◽  
Sectional Study ◽  
Cross Sectional ◽  
Rater Reliability ◽  
Pressure Injury

scholarly journals Inter-rater reliability of three most commonly used pressure ulcer risk assessment scales in clinical practice

2014 ◽  
Vol 12 (5) ◽  
pp. 590-594 ◽  
Author(s):  
Li-Hua Wang ◽  
Hong-Lin Chen ◽  
Hong-Yan Yan ◽  
Jian-Hua Gao ◽  
Fang Wang ◽  
...  
Keyword(s):  
Risk Assessment ◽  
Clinical Practice ◽  
Pressure Ulcer ◽  
Rater Reliability ◽  
Assessment Scales

VTE Risk Assessment and Prophylaxis: The Reliability of Clinical Assessment by Medical Housestaff,

Blood ◽  
2011 ◽  
Vol 118 (21) ◽  
pp. 4199-4199
Author(s):  
Monica Reddy Muppidi ◽  
Ashima Sahni ◽  
Abhimanyu Saini ◽  
Samrat Khanna ◽  
Larissa Verda ◽  
...  
Keyword(s):  
Risk Assessment ◽  
High Risk ◽  
Risk Stratification ◽  
Hospitalized Patients ◽  
Risk Assessments ◽  
Rater Reliability ◽  
Vte Prophylaxis ◽  
National Quality ◽  
Medical Inpatients ◽  
Very High

Abstract Abstract 4199 BACKGROUND: Hospital acquired venous thromboembolism (VTE) is a significant cause of mortality in hospitalized patients. The incidence of VTE may be as high as 40% in medical inpatients and is preventable in 50–75%. However, only one-half of eligible hospitalized patients receive adequate thromboprophylaxis. In response, national quality organizations and expert panels recommend a VTE risk assessment and risk-based prophylaxis for every inpatient. Point scoring systems have been proposed for risk stratification but have not been prospectively validated, and may be misleading; a recent study showed that medical residents using a point system made errors in risk stratification and choice of VTE prophylaxis. Thus, the optimal method of assessing VTE risk and whether these assessments can have adequate inter-rater reliability remains unknown. OBJECTIVES: 1. To compare the inter-rater reliability of VTE risk assessment by paired expert reviewers within the paired team and to the clinical team's assessment. 2. To evaluate the appropriateness of VTE prophylaxis administered by clinical teams compared to expert reviewer's determinations. METHODS: We performed a cross-sectional study at a 464-bed public teaching hospital. Medical patients were randomly selected and their charts abstracted by four expert housestaff reviewers (two teams of two reviewers) who had been trained through literature review, case discussion and participation in guideline development. Paired reviewers independently assessed VTE risk blinded to the other reviewer's determination using clinical data and a ‘3-bucket' model (low; moderate or high; very high). Appropriateness of prophylaxis was based on VTE risk as well as contraindications to prophylaxis. Reviewers also recorded the primary teams' VTE risk assessment and prophylaxis choices. Reviewer discrepancies were adjudicated through a third blinded review. We calculated the inter-rater reliability between paired reviewers and between reviewers and clinical teams using weighted Kappa scores (K). We recorded reasons for disagreement between reviewers and teams. RESULTS: A total of 40 charts were reviewed and analyzed for agreement on VTE risk. 36 charts were analyzed for appropriateness of VTE prophylaxis; 4 patients on therapeutic anticoagulation were excluded from this analysis. Compared to expert reviewers (E), medical teams (M) significantly underestimated VTE risk, as follows: low risk (E, 2.5% vs M, 20%); moderate to high risk (E,85% vs M,75%); very high risk (E, 12.5% vs M, 5%); P=0.004. In 11 of 12 cases of disagreement, team's assessment of VTE risk was lower than that determined by reviewers. Compared to the inter-rater reliability between experts and clinical teams, reliability was significantly better for the paired experts both for VTE risk assessment (P<0.01) and choice of prophylaxis (P<0.01). Among the 8 (22%) of patients for whom the reviewers determined VTE prophylaxis was suboptimal, for most (n=6) the method of prophylaxis was less intensive than recommended by the guidelines, and the most common reason was failure to restart prophylaxis after an invasive procedure or transfer of care. CONCLUSIONS: Our study shows that expert reviewers can assess VTE risk with a high degree of reliability. The risk assessments by clinical teams during routine clinical evaluation did not correlate well with expert risk stratification and underestimated the risk of VTE in medical inpatients. Incorrect risk assessments were common but the most frequent reasons for underutilization of VTE prophylaxis were oversights in ordering prophylaxis during care transitions or after invasive procedures. Although we trained our experts to be highly reliable in risk assessment this training cannot be generalized to most provider groups. An optimal approach to improving VTE risk assessment in clinical settings involving trainees would include real time decision support for risk assessment with linked VTE prophylaxis choices appropriate to the level of risk at the point of care. Disclosures: No relevant conflicts of interest to declare.

Inter-rater reliability of dynamic risk assessment with sexual offenders

2006 ◽  
Vol 12 (4) ◽  
pp. 439-452 ◽  
Author(s):  
Stephen D. Webster ◽  
Ruth E. Mann ◽  
Adam J. Carter ◽  
Julia Long ◽  
Rebecca J. Milner ◽  
...  
Keyword(s):  
Risk Assessment ◽  
Sexual Offenders ◽  
Rater Reliability ◽  
Dynamic Risk ◽  
Dynamic Risk Assessment

Inter-rater reliability of the glamorgan paediatric pressure ulcer risk assessment scale

2008 ◽  
Vol 20 (7) ◽  
pp. 14-19 ◽  
Author(s):  
Jane Willock ◽  
Denis Anthony ◽  
Jim Richardson
Keyword(s):  
Risk Assessment ◽  
Pressure Ulcer ◽  
Assessment Scale ◽  
Rater Reliability

Inter-rater reliability of the Bereavement Risk Assessment Tool

2011 ◽  
Vol 9 (2) ◽  
pp. 153-164 ◽  
Author(s):  
Caelin Rose ◽  
Wendy Wainwright ◽  
Michael Downing ◽  
Mary Lesperance
Keyword(s):  
Risk Assessment ◽  
Case Studies ◽  
Assessment Tool ◽  
Small Sample Size ◽  
Small Sample ◽  
Risk Assessment Tool ◽  
Risk Scores ◽  
Professional Judgment ◽  
Rater Reliability ◽  
Initial Development

AbstractObjective:The Bereavement Risk Assessment Tool (BRAT) was designed to consistently communicate information affecting bereavement outcomes; to predict the risk for difficult or complicated bereavement based on information obtained before the death; to consider resiliency as well as risk; and to assist in the efficacy and consistency of bereavement service allocation. Following initial development of the BRAT's 40 items and its clinical use, this study set out to test the BRAT for inter-rater reliability along with some basic validity measures.Method:Case studies were designed based on actual patients and families from a hospice palliative care program. Bereavement professionals were recruited via the internet. Thirty-six participants assessed BRAT items in 10 cases and then estimated one of 5 levels of risk for each case. These were compared with an expert group's assignment of risk.Results:Inter-rater reliability for the 5-level risk scores yielded a Fleiss’ kappa of 0.37 and an intra-class correlation (ICC) of 0.68 (95% CI 0.5-0.9). By collapsing scores into low and high risk groups, a kappa of 0.63 and an ICC of 0.66 (95% CI 0.5-0.9) was obtained. Participant-estimated risk scores yielded a kappa of 0.24. Although opinion varied on the tool's length, participants indicated it was well organized and easy to use with potential in assessment and allocation of bereavement services. Limitations of the study include a small sample size and the use of case studies. Limitations of the tool include the subjectivity of some items and ambiguousness of unchecked items.Significance of results:The collapsed BRAT risk levels show moderately good inter-rater reliability over clinical judgement alone. This study provides introductory evidence of a tool that can be used both prior to and following a death and, in conjunction with professional judgment, can assess the likelihood of bereavement complications.

Inter-rater reliability of the Glamorgan Paediatric Pressure Ulcer Risk Assessment Scale

2008 ◽  
Vol 20 (7) ◽  
pp. 14-19 ◽  
Author(s):  
Jane Willock ◽  
Denis Anthony ◽  
Jim Richardson
Keyword(s):  
Risk Assessment ◽  
Pressure Ulcer ◽  
Assessment Scale ◽  
Rater Reliability
Sign in / Sign up

Export Citation Format

Share Document