LO83: Effect of the transition to an entrustability scale on assessor stringency and leniency on daily encounter cards in emergency medicine

Introduction: Workplace based assessments (WBAs) are integral to emergency medicine residency training. However many biases undermine their validity, such as an assessor's personal inclination to rate learners leniently or stringently. Outlier assessors produce assessment data that may not reflect the learner's performance. Our emergency department introduced a new Daily Encounter Card (DEC) using entrustability scales in June 2018. Entrustability scales reflect the degree of supervision required for a given task, and are shown to improve assessment reliability and discrimination. It is unclear what effect they will have on assessor stringency/leniency – we hypothesize that they will reduce the number of outlier assessors. We propose a novel, simple method to identify outlying assessors in the setting of WBAs. We also examine the effect of transitioning from a norm-based assessment to an entrustability scale on the population of outlier assessors. Methods: This was a prospective pre-/post-implementation study, including all DECs completed between July 2017 and June 2019 at The Ottawa Hospital Emergency Department. For each phase, we identified outlier assessors as follows: 1. An assessor is a potential outlier if the mean of the scores they awarded was more than two standard deviations away from the mean score of all completed assessments. 2. For each assessor identified in step 1, their learners’ assessment scores were compared to the overall mean of all learners. This ensures that the assessor was not simply awarding outlying scores due to working with outlier learners. Results: 3927 and 3860 assessments were completed by 99 and 116 assessors in the pre- and post-implementation phases respectively. We identified 9 vs 5 outlier assessors (p = 0.16) in the pre- and post-implementation phases. Of these, 6 vs 0 (p = 0.01) were stringent, while 3 vs 5 (p = 0.67) were lenient. One assessor was identified as an outlier (lenient) in both phases. Conclusion: Our proposed method successfully identified outlier assessors, and could be used to identify assessors who might benefit from targeted coaching and feedback on their assessments. The transition to an entrustability scale resulted in a non-significant trend towards fewer outlier assessors. Further work is needed to identify ways to mitigate the effects of rater cognitive biases.

Prevalence and treatment of panic disorder in bipolar disorder: systematic review and meta-analysis

QuestionRecent data suggest that anxiety disorders are as often comorbid with bipolar disorder (BD) as with unipolar depression. The literature on panic disorder (PD) comorbid with BD has been systematically reviewed and subject to meta-analysis.Study selection and analysisThe Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines were thoroughly followed for literature search, selection and reporting of available evidence. The variance-stabilising Freeman-Tukey double arcsine transformation was used in the meta-analysis of prevalence estimates. Both fixed-effect and random-effects models with inverse variance method were applied to estimate summary effects for all combined studies. Heterogeneity was assessed and measured with Cochran’s Q and I2 statistics.FindingsOverall, 15 studies (n=3391) on cross-sectional prevalence and 25 independent lifetime studies (n=8226) were used to calculate pooled estimates. The overall random-effects point prevalence of PD in patients with BD, after exclusion of one potential outlier study, was 13.0% (95% CI 7.0% to 20.3%), and the overall random-effects lifetime estimate, after exclusion of one potential outlier study, was 15.5% (95% CI 11.6% to 19.9%). There were no differences in rates between BD-I and BD-II. Significant heterogeneity (I2 >95%) was found in both estimates.ConclusionsEstimates that can be drawn from published studies indicate that the prevalence of PD in patients with BD is higher than the prevalence in the general population. Comorbid PD is reportedly associated with increased risk of suicidal acts and a more severe course. There is no clear indication on how to treat comorbid PD in BD. Findings from the current meta-analysis confirm the highly prevalent comorbidity of PD with BD, implicating that in patients with BD, PD might run a more chronic course.

Analysis Of Geodetic Network Established Inside The Dobšinská Ice Cave Space / Analýza Geodetickej Siete Zriadenej V Priestoroch Dobšinskej Ľadovej Jaskyne

The present article summarizes the progress and results of geodetic works during the construction of a geodetic network inside the Dobšinská Ice Cave underground space to monitor temporal and spatial changes in its ice filling. In order to objectively evaluate the changes, parameter estimations of the first- and second-order of the geodetic network from the set of field geodetic measurements were provided, and a robust analysis of the network was applied in terms of the assessment of impacts of potential outlier measurements on the network geometry

Use of Medicare Claims to Identify US Hospitals with a High Rate of Surgical Site Infection after Hip Arthroplasty

Objective.To assess the ability of Medicare claims to identify US hospitals with high rates of surgical site infection (SSI) after hip arthroplasty.Design.Retrospective cohort study.Setting.Acute care US hospitals.Participants.Fee-for-service Medicare patients 65 years of age and older who underwent hip arthroplasty in US hospitals from 2005 through 2007.Methods.Hospital rankings were derived from claims codes suggestive of SSI, adjusted for age, sex, and comorbidities, while using generalized linear mixed models to account for hospital volume. Medical records were obtained for validation of infection on a random sample of patients from hospitals ranked in the best and worst deciles of performance. We then calculated the risk-adjusted odds of developing a chart-confirmed SSI after hip arthroplasty in hospitals ranked by claims into worst- versus best-performing deciles.Results.Among 524,892 eligible Medicare patients who underwent hip arthroplasty at 3,296 US hospitals, a patient who underwent surgery in a hospital ranked in the worst-performing decile based on claims-based evidence of SSI had 2.9-fold higher odds of developing a chart-confirmed SSI relative to a patient with the same age, sex, and comorbidities in a hospital ranked in the best-performing decile (95% confidence interval, 2.2-3.7).Conclusions.Medicare claims successfully distinguished between hospitals with high and low SSI rates following hip arthroplasty. These claims can identify potential outlier hospitals that merit further evaluation. This strategy can also be used to validate the completeness of public reporting of SSI.