scholarly journals MP16: Oral case presentation: evaluation of a novel curriculum and development of a competency-based assessment tool in Emergency Medicine

CJEM ◽  
2020 ◽  
Vol 22 (S1) ◽  
pp. S48-S48
Author(s):  
T. Wawrykow ◽  
T. McColl ◽  
A. Velji ◽  
M. Chan
Keyword(s):  
Emergency Medicine ◽  
Medical Students ◽  
Blended Learning ◽  
Assessment Tool ◽  
Intervention Group ◽  
Emergency Setting ◽  
Case Presentation ◽  
Canadian Association ◽  
Competency Based ◽  
Competency Based Assessment

Introduction: The oral case presentation is recognized as a core educational and patient care activity but has not been well studied in the emergency setting. The objectives of this study are: 1) to develop a competency-based assessment tool to formally evaluate the emergency medicine oral case presentation (EM-OCP) competency of medical students and ‘transition to discipline’ residents, and 2) to develop, implement and evaluate a curriculum to enhance oral case presentation (OCP) communication skills in the emergency medicine (EM) setting. Methods: Using data from a literature review, a Canadian Association of Emergency Physicians national survey, and local focus groups, the authors designed an OCP framework, blended learning curriculum, and EM-OCP assessment tool. Ninety-six clerkship students were randomly assigned to receive either the control, the standard clerkship curriculum, or intervention, the blended learning curriculum. At the beginning of their emergency medicine rotation, learners completed a pre-test using a standardized patient (SP) case to assess their baseline OCP skills. The intervention group then completed the EM-OCP curriculum. All students completed post-tests with a different SP at the end of the six-week EM rotation. Audio-recordings of pre and post-tests were evaluated using the assessment tool by two blinded evaluators. Results: Using the Kruskal-Wallis test, all students demonstrated improvement in EM-OCP skills between their pre-test and post-test, however, those who received the blended learning curriculum showed significantly greater improvement in synthesis of information (p = 0.044), management (p = 0.006) and overall entrustment decision score (p = 0.000). Conclusion: Implementation of a novel EM-OCP curriculum resulted in more effective communication and higher entrustment scores. This curriculum could improve OCP performance not only in emergency medicine settings but also across specialties where medical students and residents must manage critical patients.

scholarly journals P057: Performance of a national simulation-based resuscitation OSCE for emergency medicine trainees

CJEM ◽  
2016 ◽  
Vol 18 (S1) ◽  
pp. S97-S98 ◽  
Author(s):  
C. Hagel ◽  
A.K. Hall ◽  
D. Klinger ◽  
G. McNeil ◽  
D. Dagnone
Keyword(s):  
Emergency Medicine ◽  
Large Scale ◽  
Assessment Tool ◽  
Postgraduate Medical Education ◽  
Perceived Realism ◽  
Simulation Based ◽  
Competency Based ◽  
Generalizability Study ◽  
Graduate Trainees ◽  
Competency Based Assessment

Introduction: The use of high-fidelity simulation is emerging as an effective method for competency-based assessment in postgraduate medical education. We have previously reported the development of the Queen’s Simulation Assessment Tool (QSAT), for use in simulation-based Objective Structured Clinical Examinations (OSCEs) for Emergency Medicine (EM) trainees. We aimed to demonstrate the feasibility and present an argument for the validity of a simulation-based OSCE utilizing the QSAT with EM residents from multiple Canadian training sites. Methods: EM post-graduate trainees (PGY 2-5) from 9 Canadian EM training programs participated in an 8-station simulation-based resuscitation OSCE at Queen’s University in Kingston, ON. Each station was scored by a single trained rater from a group of 9 expert Canadian EM physicians. Raters utilized a station-specific QSAT and provided an Entrustment Score. A post-examination questionnaire was administered to the trainees to quantify perceived realism, comfort and educational impact. Statistical analyses included analysis of variance to measure the discriminatory capabilities and a generalizability study to examine the sources of variability in the scores. Results: EM postgraduate trainees (N=36) participated in the study. Discriminatory validity was strong, with senior trainees (PGY4-5) outperforming junior trainees (PGY2-3) in 6 of 8 scenarios and in aggregated QSAT and Entrustment Scores across all 8 stations (p<0.01). Generalizability studies found the largest sources of random variability was due to the trainee by station interaction and the error term, with a G coefficient of 0.84. Resident trainees reported reasonable comfort being assessed in the simulation environment (3.6/5), indicated significant perceived realism (4.1/5), and found the OSCE valuable to their learning (4.8/5). Conclusion: Overall, this study demonstrates that a large-scale simulation-based EM resuscitation OSCE is feasible, and an argument has been presented for the validity of such an examination. The incorporation of simulation or a simulation-based OSCE in the national certification process in EM may help to satisfy the increased demand for competency-based assessment required by the Royal College of Physicians & Surgeons of Canada’s Competency by Design transition.

scholarly journals Hot Seat Diagnosis: Competency-Based Tool Is Superior to Time-Based Tool for the Formative In-Service Assessment of Pathology Trainees

2021 ◽  
Author(s):  
Rachel Han ◽  
Julia Keith ◽  
Elzbieta Slodkowska ◽  
Sharon Nofech-Mozes ◽  
Bojana Djordjevic ◽  
...  
Keyword(s):  
Assessment Tool ◽  
Training Evaluation ◽  
Assessment Tools ◽  
Assessment Scale ◽  
Clinical Implications ◽  
Competency Based Medical Education ◽  
Evaluation Report ◽  
Competency Based ◽  
Diagnosis Tool ◽  
Competency Based Assessment

Context.— Competency-based medical education relies on frequent formative in-service assessments to ascertain trainee progression. Currently at our institution, trainees receive a summative end-of-rotation In-Training Evaluation Report based on feedback collected from staff pathologists. There is no method of simulating report sign-out. Objective.— To develop a formative in-service assessment tool that is able to simulate report sign-out and provide case-by-case feedback to trainees. Further, to compare time- versus competency-based assessment models. Design.— Twenty-one pathology trainees were assessed for 20 months. Hot Seat Diagnosis by trainees and trainee assessment by pathologists were recorded in the Laboratory Information System. In the first iteration, trainees were assessed by using a time-based assessment scale on their ability to diagnose, report, use ancillary testings, comment on clinical implications, provide intraoperative consultation and/or gross cases. The second iteration used a competency-based assessment scale. Trainees and pathologists completed surveys on the effectiveness of the In-Training Evaluation Report versus the Hot Seat Diagnosis tool. Results.— Scores from both iterations correlated significantly with other assessment tools including the Resident In-Service Examination (r = 0.93, P = .04 and r = 0.87, P = .03). The competency-based model was better able to demonstrate improvement over time and stratify junior versus senior trainees than the time-based model. Trainees and pathologists rated Hot Seat Diagnosis as significantly more objective, detailed, and timely than the In-Training Evaluation Report, and effective at simulating report sign-out. Conclusions.— Hot Seat Diagnosis is an effective tool for the formative in-service assessment of pathology trainees and simulation of report sign-out, with the competency-based model outperforming the time-based model.

scholarly journals Competency-based simulation assessment of resuscitation skills in emergency medicine postgraduate trainees – a Canadian multi-centred study

2016 ◽  
Vol 7 (1) ◽  
pp. e57-e67 ◽  
Author(s):  
J. Damon Dagnone ◽  
Andrew K. Hall ◽  
Stefanie Sebok-Syer ◽  
Don Klinger ◽  
Karen Woolfrey ◽  
...  
Keyword(s):  
Emergency Medicine ◽  
Assessment Tool ◽  
Postgraduate Medical Education ◽  
Intraclass Correlation ◽  
Correlation Coefficients ◽  
High Fidelity ◽  
Rater Reliability ◽  
Simulation Based ◽  
Competency Based ◽  
Generalizability Study

Background: The use of high-fidelity simulation is emerging as a desirable method for competency-based assessment in postgraduate medical education. We aimed to demonstrate the feasibility and validity of a multi-centre simulation-based Objective Structured Clinical Examination (OSCE) of resuscitation competence with Canadian Emergency Medicine (EM) trainees.Method: EM postgraduate trainees (n=98) from five Canadian academic centres participated in a high fidelity, 3-station simulation-based OSCE.  Expert panels of three emergency physicians evaluated trainee performances at each centre using the Queen’s Simulation Assessment Tool (QSAT).  Intraclass correlation coefficients were used to measure the inter-rater reliability, and analysis of variance was used to measure the discriminatory validity of each scenario.  A fully crossed generalizability study was also conducted for each examination centre.   Results: Inter-rater reliability in four of the five centres was strong with a median absolute intraclass correlation coefficient (ICC) across centres and scenarios of 0.89 [0.65-0.97]. Discriminatory validity was also strong (p < 0.001 for scenarios 1 and 3; p < 0.05 for scenario 2). Generalizability studies found significant variations at two of the study centres.Conclusions: This study demonstrates the successful pilot administration of a multi-centre, 3-station simulation-based OSCE for the assessment of resuscitation competence in post-graduate Emergency Medicine trainees.

scholarly journals Outcomes in the age of competency-based medical education: Recommendations for emergency medicine training in Canada from the 2019 symposium of academic emergency physicians

CJEM ◽  
2020 ◽  
Vol 22 (2) ◽  
pp. 204-214 ◽  
Author(s):  
Teresa M. Chan ◽  
Quinten S. Paterson ◽  
Andrew K. Hall ◽  
Fareen Zaver ◽  
Robert A. Woods ◽  
...  
Keyword(s):  
Emergency Medicine ◽  
Medical Education ◽  
Clinical Outcomes ◽  
Online Survey ◽  
Emergency Physicians ◽  
Competency Based Medical Education ◽  
Canadian Association ◽  
National Implementation ◽  
Competency Based ◽  
The Impact

ABSTRACTObjectivesThe national implementation of competency-based medical education (CBME) has prompted an increased interest in identifying and tracking clinical and educational outcomes for emergency medicine training programs. For the 2019 Canadian Association of Emergency Physicians (CAEP) Academic Symposium, we developed recommendations for measuring outcomes in emergency medicine training in the context of CBME to assist educational leaders and systems designers in program evaluation.MethodsWe conducted a three-phase study to generate educational and clinical outcomes for emergency medicine (EM) education in Canada. First, we elicited expert and community perspectives on the best educational and clinical outcomes through a structured consultation process using a targeted online survey. We then qualitatively analyzed these responses to generate a list of suggested outcomes. Last, we presented these outcomes to a diverse assembly of educators, trainees, and clinicians at the CAEP Academic Symposium for feedback and endorsement through a voting process.ConclusionAcademic Symposium attendees endorsed the measurement and linkage of CBME educational and clinical outcomes. Twenty-five outcomes (15 educational, 10 clinical) were derived from the qualitative analysis of the survey results and the most important short- and long-term outcomes (both educational and clinical) were identified. These outcomes can be used to help measure the impact of CBME on the practice of Emergency Medicine in Canada to ensure that it meets both trainee and patient needs.

scholarly journals Education First: Promoting LGBT+ Friendly Healthcare with a Competency-Based Course and Game-Based Teaching

2019 ◽  
Vol 17 (1) ◽  
pp. 107
Author(s):  
Hsing-Chen Yang
Keyword(s):  
Medical Education ◽  
Medical Students ◽  
Assessment Tool ◽  
Assessment Tools ◽  
Learning Goals ◽  
Case Study Methodology ◽  
Teaching Activities ◽  
Specific Teaching ◽  
Competency Based ◽  
Gender Knowledge

How, apart from by conveying professional knowledge, can university medical education nurture and improve the gender competency of medical students and thereby create an LGBT+ friendly healthcare environment? This study explored the use of game-based teaching activities in competency-based teaching from the perspective of competency-based medical education (CBME) and employed a qualitative case-study methodology. We designed an LGBT+ Health and Medical Care course in a medical school. Feedback was collected from two teachers and 19 medical students using in-depth interviews and thematic analysis was used to analyze the collected data. The findings of this study were as follows: (1) Games encouraged student participation and benefited gender knowledge transmission and transformation through competency learning, and (2) games embodied the idea of assessment as learning. The enjoyable feeling of pressure from playing games motivated students to learn. Using games as both a teaching activity and an assessment tool provided the assessment and instant feedback required in the CBME learning process. Game-based teaching successfully guided medical students to learn about gender and achieve the learning goals of integrating knowledge, attitudes, and skills. To fully implement CBME using games as teaching methods, teaching activities, learning tasks, and assessment tools, teachers must improve their teaching competency. This study revealed that leading discussions and designing curricula are key in the implementation of gender competency-based education; in particular, the ability to lead discussions is the core factor. Game-based gender competency education for medical students can be facilitated with discussions that reinforce learning outcomes to achieve the objectives of gender equality education and LGBT+ friendly healthcare. The results of this study indicated that game-based CBME with specific teaching strategies was an effective method of nurturing the gender competency of medical students. The consequent integration of gender competency into medical education could achieve the goal of LGBT+ friendly healthcare.

scholarly journals UME-to-GME PandEMonium in COVID-19: Large-Scale Implementation of a Virtual ACGME Milestone-Based Curriculum for Senior Medical Students Matched Into Emergency Medicine

2021 ◽  
Vol 13 (6) ◽  
pp. 848-857
Author(s):  
Mary E. McLean ◽  
Adrian A. Cotarelo ◽  
Thomas A. Huls ◽  
Abbas Husain ◽  
Emily A. Hillman ◽  
...  
Keyword(s):  
Emergency Medicine ◽  
Medical Education ◽  
Medical Students ◽  
Large Scale ◽  
The United States ◽  
Negative Effects ◽  
Residency Programs ◽  
Perceived Preparedness ◽  
Competency Based ◽  
June July

ABSTRACT Background The COVID-19 pandemic displaced newly matched emergency medicine “pre-interns” from in-person educational experiences at the end of medical school. This called for novel remote teaching modalities. Objective This study assesses effectiveness of a multisite Accreditation Council for Graduate Medical Education (ACGME) sub-competency-based curricular implementation on Slack during the first wave of the COVID-19 pandemic in the United States. Methods Emergency medicine residency programs were recruited via national organization listservs. Programs designated instructors to manage communications and teaching for the senior medical students who had matched to their programs (pre-interns) in spring/summer 2020. Pre- and post-surveys of trainees and instructors assessed perceived preparedness for residency, perceived effectiveness of common virtual educational modalities, and concern for the pandemic's effects on medical education utilizing a Likert scale of 1 (very unconcerned) to 5 (very concerned). Data were analyzed using descriptive statistics and the t test. Results Of 276 possible residency programs, 28 enrolled. Of 324 possible pre-interns, 297 (91.7%) completed pre-surveys in April/May and 249 (76.9%) completed post-surveys in June/July. The median weeks since performing a physical examination was 8 (IQR 7–12), since attending in-person didactics was 10 (IQR 8–15) and of rotation displacement was 4 (IQR 2–6). Perceived preparedness increased both overall and for 14 of 21 ACGME Milestone topics taught. Instructors reported higher mean concern (4.32, 95% CI 4.23–4.41) than pre-interns (2.88, 95% CI 2.74–3.02) regarding the pandemic's negative effects on medical education. Conclusions Pre-interns reported improvements in residency preparedness after participating in this ACGME sub-competency-based curriculum on Slack.

Sign in / Sign up

Export Citation Format

Share Document