scholarly journals MP17: Evaluation of a national competency-based assessment system in emergency medicine: A CanDREAM study

CJEM ◽  
2020 ◽  
Vol 22 (S1) ◽  
pp. S48-S48
Author(s):  
B. Thoma ◽  
A. Hall ◽  
K. Clark ◽  
N. Meshkat ◽  
W. Cheung ◽  
...  
Keyword(s):  
Emergency Medicine ◽  
Outcome Evaluation ◽  
Assessment System ◽  
Professional Activity ◽  
National Guidelines ◽  
National Assessment ◽  
Competency Based Medical Education ◽  
Level Data ◽  
Starting Point ◽  
Competency Based

Introduction: In 2018, Canadian postgraduate specialist Emergency Medicine (EM) programs began implementing a competency-based medical education (CBME) assessment system. To support improvement of this assessment program, we sought to evaluate its short-term educational outcomes nationally and within individual programs. Methods: Program-level data from the 2018 resident cohort were amalgamated and analyzed. The number of Entrustable Professional Activity (EPA) assessments (overall and for each EPA) and the timing of resident promotion through program stages was compared between programs and to the guidelines provided by the national EM specialty committee. Total EPA observations from each program were correlated with the number of EM and pediatric EM rotations. Results: Data from 15 of 17 (88.2%) EM programs containing 9,842 EPA observations from 68 of the 77 (88.3%) Canadian EM specialist residents in the 2018 cohort were analyzed. The average number of EPAs observed per resident in each program varied from 92.5 to 229.6 and correlated strongly with the number of blocks spent on EM and pediatric EM (r = 0.83, p < 0.001). Relative to the guidelines outlined by the specialty committee, residents were promoted later than expected and with fewer EPA observations than suggested. Conclusion: We present a new approach to the amalgamation of national and program-level assessment data. There was demonstrable variation in both EPA-based assessment numbers and promotion timelines between programs and with national guidelines. This evaluation data will inform the revision of local programs and national guidelines and serve as a starting point for further reaching outcome evaluation. This process could be replicated by other national assessment programs.

scholarly journals Evaluation of a National Competency-Based Assessment System in Emergency Medicine: A CanDREAM Study

2020 ◽  
Vol 12 (4) ◽  
pp. 425-434
Author(s):  
Brent Thoma ◽  
Andrew K. Hall ◽  
Kevin Clark ◽  
Nazanin Meshkat ◽  
Warren J. Cheung ◽  
...  
Keyword(s):  
Emergency Medicine ◽  
Assessment System ◽  
Professional Activity ◽  
National Guidelines ◽  
Competency Based Medical Education ◽  
Second Stage ◽  
Assessment Program ◽  
Level Data ◽  
Competency Based ◽  
Using Data

ABSTRACT Background In 2018, Canadian postgraduate emergency medicine (EM) programs began implementing a competency-based medical education (CBME) assessment program. Studies evaluating these programs have focused on broad outcomes using data from national bodies and lack data to support program-specific improvement. Objective We evaluated the implementation of a CBME assessment program within and across programs to identify successes and opportunities for improvement at the local and national levels. Methods Program-level data from the 2018 resident cohort were amalgamated and analyzed. The number of entrustable professional activity (EPA) assessments (overall and for each EPA) and the timing of resident promotion through program stages were compared between programs and to the guidelines provided by the national EM specialty committee. Total EPA observations from each program were correlated with the number of EM and pediatric EM rotations. Results Data from 15 of 17 (88%) programs containing 9842 EPA observations from 68 of 77 (88%) EM residents in the 2018 cohort were analyzed. Average numbers of EPAs observed per resident in each program varied from 92.5 to 229.6, correlating with the number of blocks spent on EM and pediatric EM (r = 0.83, P &lt; .001). Relative to the specialty committee's guidelines, residents were promoted later than expected (eg, one-third of residents had a 2-month delay to promotion from the first to second stage) and with fewer EPA observations than suggested. Conclusions There was demonstrable variation in EPA-based assessment numbers and promotion timelines between programs and with national guidelines.

scholarly journals P101: The development of entrustable professional activity reference cards to support the implementation of Competence by Design in emergency medicine

CJEM ◽  
2019 ◽  
Vol 21 (S1) ◽  
pp. S100
Author(s):  
E. Stoneham ◽  
L. Witt ◽  
Q. Paterson ◽  
L. Martin ◽  
B. Thoma
Keyword(s):  
Emergency Medicine ◽  
Low Cost ◽  
Intervention Mapping ◽  
Assessment System ◽  
Professional Activity ◽  
Residency Programs ◽  
Professional Activities ◽  
Competency Based ◽  
Curriculum Map ◽  
User Friendly

Innovation Concept: Competence by Design (CBD) was implemented nationally for Emergency Medicine (EM) residents beginning training in 2018. One challenge is the need to introduce residents to Entrustable Professional Activities (EPAs) that are assessed across numerous clinical rotations. The Royal College's resources detail these requirements, but do not map them to specific rotations or present them in a succinct format. This is problematic as trainees are less likely to succeed when expectations are unclear. We identified a need to create practical resources that residents can use at the bedside. Methods: We followed an intervention mapping framework to design two practical, user-friendly, low-cost, aesthetically pleasing resources that could be used by residents and observers at the bedside to facilitate competency-based assessment. Curriculum, Tool or Material: First, we designed a set of rotation- and stage-specific EPA reference cards for the use of residents and observers at the bedside. These cards list EPAs and clinical presentations likely to be encountered during various stages of training and on certain rotations. Second, we developed a curriculum board to organize the EPA reference cards by stage based upon our program's curriculum map. The curriculum board allows residents to view the program's curriculum map and the EPAs associated with each clinical rotation at a glance. It also contains hooks to hang and store extra cards in an organized manner. Conclusion: We believe that these practical and inexpensive tools facilitated our residency program's transition to competency-based EPA assessments. Anecdotally, the residents are using the cards and completing the suggested rotation-specific EPAs. We hope that the reference cards and curriculum board will be successfully incorporated into other residency programs to facilitate the introduction of their EPA-based CBD assessment system.

scholarly journals Building a Shared Mental Model of Competence Across the Continuum: Trainee Perceptions of Subinternships for Residency Preparation

2021 ◽  
Vol 8 ◽  
pp. 238212052110633
Author(s):  
Johannah M. Scheurer ◽  
Cynthia Davey ◽  
Anne G. Pereira ◽  
Andrew P. J. Olson
Keyword(s):  
Medical Education ◽  
Response Rate ◽  
Professional Activity ◽  
University Of Minnesota ◽  
Competency Based Medical Education ◽  
Chi Squared ◽  
Competency Based ◽  
Kappa Agreement ◽  
Physician Competency ◽  
The Continuum

INTRODUCTION Toward a vision of competency-based medical education (CBME) spanning the undergraduate to graduate medical education (GME) continuum, University of Minnesota Medical School (UMMS) developed the Subinternship in Critical Care (SICC) offered across specialties and sites. Explicit course objectives and assessments focus on internship preparedness, emphasizing direct observation of handovers (Core Entrustable Professional Activity, “EPA,” 8) and cross-cover duties (EPA 10). METHODS To evaluate students’ perceptions of the SICC's and other clerkships’ effectiveness toward internship preparedness, all 2016 and 2017 UMMS graduates in GME training ( n = 440) were surveyed regarding skill development and assessment among Core EPAs 1, 4, 6, 8, 9, 10. Analysis included descriptive statistics plus chi-squared and Kappa agreement tests. RESULTS Respondents ( n = 147, response rate 33%) rated the SICC as a rotation during which they gained most competence among EPAs both more (#4, 57% rated important; #8, 75%; #10, 70%) and less explicit (#6, 53%; #9, 69%) per rotation objectives. Assessments of EPA 8 (80% rated important) and 10 (76%) were frequently perceived as important toward residency preparedness. Agreement between importance of EPA development and assessment was moderate (Kappa = 0.40-0.59, all surveyed EPAs). CONCLUSIONS Graduates’ perceptions support the SICC's educational utility and assessments. Based on this and other insight from the SICC, the authors propose implications toward collectively envisioning the continuum of physician competency.

scholarly journals Adapting Feedback to Individual Residents: An Examination of Preceptor Challenges and Approaches

2018 ◽  
Vol 10 (2) ◽  
pp. 168-175 ◽  
Author(s):  
Amanda Roze des Ordons ◽  
Adam Cheng ◽  
Jonathan Gaudet ◽  
James Downar ◽  
Jocelyn Lockyer
Keyword(s):  
Medical Education ◽  
Professional Activity ◽  
Framework Analysis ◽  
Competency Based Medical Education ◽  
Performance Levels ◽  
Skill Sets ◽  
Entrustable Professional Activity ◽  
Competency Based ◽  
Individualize Feedback ◽  
Insight Into

ABSTRACT Background  Feedback conversations between preceptors and residents usually occur in closed settings. Little is known about how preceptors address the challenges posed by residents with different skill sets, performance levels, and personal contexts. Objective  This study explored the challenges that preceptors experienced and approaches taken in adapting feedback conversations to individual residents. Methods  In 2015, 18 preceptors participated in feedback simulations portraying residents with variations in skill, insight, confidence, and distress, followed by debriefing of the feedback conversation with a facilitator. These interactions were recorded, transcribed, and analyzed using thematic and framework analysis. Results  The preceptors encountered common challenges with feedback conversations, including uncertainty in how to individualize feedback to residents and how to navigate tensions between resident- and preceptor-identified goals. Preceptors questioned their ability to enhance skills for highly performing residents, whether they could be directive when residents had insight gaps, how they could reframe the perceptions of the overly confident resident, and whether they should offer support to emotionally distressed residents or provide feedback about performance. Preceptors adapted their approach to feedback, drawing on techniques of coaching for highly performing residents, directing for residents with insight gaps, mediation with overly confident residents, and mentoring with emotionally distressed residents. Conclusions  Examining the feedback challenges preceptors encounter and the approaches taken to adapt feedback to individual residents can provide insight into how preceptors meet the challenges of competency-based medical education, in which frequent, focused feedback is essential for residents to achieve educational milestones and entrustable professional activity expectations.

Faculty development in the age of competency-based medical education: A needs assessment of Canadian emergency medicine faculty and senior trainees

CJEM ◽  
2019 ◽  
Vol 21 (4) ◽  
pp. 527-534 ◽  
Author(s):  
Alexandra Stefan ◽  
Justin N. Hall ◽  
Jonathan Sherbino ◽  
Teresa M. Chan
Keyword(s):  
Emergency Medicine ◽  
Medical Education ◽  
Faculty Development ◽  
Patient Care ◽  
Needs Assessment ◽  
Direct Observation ◽  
Senior Resident ◽  
Cross Sectional ◽  
Competency Based Medical Education ◽  
Competency Based

ABSTRACTObjectivesThe Royal College of Physicians and Surgeons of Canada (RCPSC) emergency medicine (EM) programs transitioned to the Competence by Design training framework in July 2018. Prior to this transition, a nation-wide survey was conducted to gain a better understanding of EM faculty and senior resident attitudes towards the implementation of this new program of assessment.MethodsA multi-site, cross-sectional needs assessment survey was conducted. We aimed to document perceptions about competency-based medical education, attitudes towards implementation, perceived/prompted/unperceived faculty development needs. EM faculty and senior residents were nominated by program directors across RCPSC EM programs. Simple descriptive statistics were used to analyse the data.ResultsBetween February and April 2018, 47 participants completed the survey (58.8% response rate). Most respondents (89.4%) thought learners should receive feedback during every shift; 55.3% felt that they provided adequate feedback. Many respondents (78.7%) felt that the ED would allow for direct observation, and most (91.5%) participants were confident that they could incorporate workplace-based assessments (WBAs). Although a fair number of respondents (44.7%) felt that Competence by Design would not impact patient care, some (17.0%) were worried that it may negatively impact it. Perceived faculty development priorities included feedback delivery, completing WBAs, and resident promotion decisions.ConclusionsRCPSC EM faculty have positive attitudes towards competency-based medical education-relevant concepts such as feedback and opportunities for direct observation via WBAs. Perceived threats to Competence by Design implementation included concerns that patient care and trainee education might be negatively impacted. Faculty development should concentrate on further developing supervisors’ teaching skills, focusing on feedback using WBAs.

scholarly journals Sustainability of physical exam skills in a resident-led curriculum in a large internal medicine program with competency based medical education

2018 ◽  
Vol 9 (4) ◽  
pp. e78-92 ◽  
Author(s):  
Don Thiwanka Wijeratne ◽  
Siddhartha Srivastava ◽  
Barry Chan ◽  
Wilma Hopman ◽  
Benjamin Thomson
Keyword(s):  
Internal Medicine ◽  
Medical Education ◽  
Physical Examination ◽  
Peer Assessment ◽  
Physical Exam ◽  
Professional Activity ◽  
Faculty Assessment ◽  
Competency Based Medical Education ◽  
Competency Based ◽  
Faculty Assessments

Background: Competency Based Medical Education (CBME) designates physical examination competency as an Entrustable Professional Activity (EPA). Considerable concern persists regarding the increased time burden CBME may place on educators. We developed a novel physical examination curriculum that shifted the burden of physical examination case preparation and performance assessment from faculty to residents. Our first objective was to determine if participation led to sustainable improvements in physical examination skills. The second objective was to determine if resident peer assessment was comparable to faculty assessment.    Methods: We selected physical exam case topics based on the Objectives of Training in the Specialty of Internal Medicine as prescribed by the Royal College of Physicians and Surgeons of Canada. Internal Medicine residents compiled evidence-based physical exam checklists that faculty reviewed before distribution to all learners. Physical exam practice sessions with whole-group demonstration followed by small-group practice sessions were performed weekly. We evaluated this pilot curriculum with a formative OSCE, during which a resident peer and a faculty member simultaneously observed and assessed examinee performance by .Results: Participation in the novel curriculum practice sessions improved OSCE performance (faculty score mean 78.96 vs. 62.50, p<0.05). Peer assessment overestimated faculty scores (76.2 vs. 65.7, p<0.001), but peer and faculty assessments were highly correlated (R2 = 0.73 (95% CI 0.50-0.87).Conclusion: This novel physical examination curriculum leads to sustainable improvement of physical examination skills. Peer assessment correlated well with the gold standard faculty assessment. This resident-led physical examination curriculum enhanced physical examination skills in a CBME environment, with minimal time commitment from faculty members.

scholarly journals EFFICACY OF COMPETENCY-BASED APPROACH IN TEACHING PROCESS OF PHYSICAL EDUCATION IN A UNIVERSITY

2020 ◽  
pp. 73-78
Author(s):  
O. A. Melnikova ◽  
I. N. Sheveleva
Keyword(s):  
Physical Education ◽  
Questionnaire Survey ◽  
Performance Test ◽  
Educational Process ◽  
Assessment System ◽  
Professional Activity ◽  
Educational Training ◽  
Sports Activity ◽  
Competency Based ◽  
And Performance

The study addresses the efficacy of the competency-based approach in physical education on the basis of the developed model of universal competency (UC-7) formation. The experimental work at the informational stage involved a questionnaire survey, a physical performance test, studies of morphological and functional indicators and psychophysical state. The organizational-activity stage included a program of methodological-practical and educational-training sessions, based on the personality-oriented approach to physical culture and sports activity and aimed at the formation of special knowledge, abilities and skills that contribute to maintaining the proper level of physical fitness and to ensure full-fledged social and professional activity. The control stage included a questionnaire survey and tests aimed at determining the psycho-emotional state, which allowed determining the level of competency formation in students according to the developed assessment system. The competency-based approach was introduced to the physical educational process based on the universal competency formation model. The educational process was reorganized pursuant to new pedagogical conditions. The results were the improvement of students’ physical activity, fitness and performance; the indicators of pulmonary function improved; and the level of personal and situational anxiety and depression in students decreased.

scholarly journals An Entrustable Professional Activity Addressing Racism and Pediatric Health Inequities

PEDIATRICS ◽  
2022 ◽  
Author(s):  
Ndidi I. Unaka ◽  
Ariel Winn ◽  
Adiaha Spinks-Franklin ◽  
Patricia Poitevien ◽  
Franklin Trimm ◽  
...  
Keyword(s):  
Medical Education ◽  
Health Care Systems ◽  
Health Inequities ◽  
Professional Activity ◽  
Competency Based Medical Education ◽  
Pediatric Health ◽  
Professional Activities ◽  
Care Systems ◽  
Competency Based ◽  
Pediatric Populations

Racism and discrimination are the root of many pediatric health inequities and are well described in the literature. Despite the pervasiveness of pediatric health inequities, we have failed to adequately educate and prepare general pediatricians and pediatric subspecialists to address them. Deficiencies within education across the entire continuum and in our health care systems as a whole contribute to health inequities in unacceptable ways. To address these deficiencies, the field of pediatrics, along with other specialties, has been on a journey toward a more competency-based approach to education and assessment, and the framework created for the future is built on entrustable professional activities (EPAs). Competency-based medical education is one approach to addressing the deficiencies within graduate medical education and across the continuum by allowing educators to focus on the desired equitable patient outcomes and then develop an approach to teaching and assessing the tasks, knowledge, skills, and attitudes needed to achieve the goal of optimal, equitable patient care. To that end, we describe the development and content of a revised EPA entitled: Use of Population Health Strategies and Quality Improvement Methods to Promote Health and Address Racism, Discrimination, and Other Contributors to Inequities Among Pediatric Populations. We also highlight the ways in which this EPA can be used to inform curricula, assessments, professional development, organizational systems, and culture change.

scholarly journals Teaching and assessing intra-operative consultations in competency-based medical education: development of a workplace-based assessment instrument

2021 ◽  
Author(s):  
Marcio M. Gomes ◽  
David Driman ◽  
Yoon Soo Park ◽  
Timothy J. Wood ◽  
Rachel Yudkowsky ◽  
...  
Keyword(s):  
Medical Education ◽  
Assessment Tools ◽  
Assessment Instrument ◽  
Professional Activity ◽  
Cultural Issues ◽  
Education Development ◽  
Competency Based Medical Education ◽  
Workplace Based Assessment ◽  
Competency Based ◽  
Validity Theory

AbstractCompetency-based medical education (CBME) is being implemented worldwide. In CMBE, residency training is designed around competencies required for unsupervised practice and use entrustable professional activities (EPAs) as workplace “units of assessment”. Well-designed workplace-based assessment (WBA) tools are required to document competence of trainees in authentic clinical environments. In this study, we developed a WBA instrument to assess residents’ performance of intra-operative pathology consultations and conducted a validity investigation. The entrustment-aligned pathology assessment instrument for intra-operative consultations (EPA-IC) was developed through a national iterative consultation and used clinical supervisors to assess residents’ performance at an anatomical pathology program. Psychometric analyses and focus groups were conducted to explore the sources of evidence using modern validity theory: content, response process, internal structure, relations to other variables, and consequences of assessment. The content was considered appropriate, the assessment was feasible and acceptable by residents and supervisors, and it had a positive educational impact by improving performance of intra-operative consultations and feedback to learners. The results had low reliability, which seemed to be related to assessment biases, and supervisors were reluctant to fully entrust trainees due to cultural issues. With CBME implementation, new workplace-based assessment tools are needed in pathology. In this study, we showcased the development of the first instrument for assessing resident’s performance of a prototypical entrustable professional activity in pathology using modern education principles and validity theory.

scholarly journals COMPETENCY-BASED APPROACH IN THE FUTURE TEACHER OF BIOLOGY TRAINING SYSTEM

2018 ◽  
pp. 49-52
Author(s):  
O. VOROBIOVA
Keyword(s):  
Training System ◽  
Educational Process ◽  
Theory And Practice ◽  
Future Teacher ◽  
General Idea ◽  
Professional Activity ◽  
Research Activity ◽  
Starting Point ◽  
The Future ◽  
Competency Based

Іn the article the modern scientific researches on building and development of key competencies in theory and practice of pedagogical science are analyzed. The peculiarities of the professional activity of the teacher of biology through the interrelation of his/her informative, developmental, orientation, mobilizing, constructive, communicative, and organizational and research functions are researched.The analysis of modern approaches in the preparation of the future teacher of biology made it possible to determine leading functions in the educational process of a secondary school: informative, developmental, orientation, mobilization, constructive, communicative, and organizational and research. All functions are interrelated; they complement and refine each other.A large number of researchers prove that the competency-based approach most completely reflects the modern processes in European countries.Taking into account the world experience and needs of the development of the Ukrainian school in the domestic pedagogy, three types of competencies characterizing the results of learning on the basis of a competency-based approach are recognized: key, interdisciplinary and knowledge of subject.The notion of "competency-based approach" refers to the orientation of the educational process on the building and development of the key (basic, main) and substantive competencies of the individual. The result of this process will be the formation of a general competence (professionalism) of a person, which is the integrity of key and substantive competencies, an integrated personality trait. Such a characteristic should be formed in the process of learning and  contain knowledge, skills, attitudes, experience, values and behavioural models of personality.The competency-based approach takes special place in the process of building and development of competencies of the future specialist in the system of higher education. In the scientific sense, the concept of "approach" is interpreted as the starting point for forming the basis of research activity. The general idea of a competency-based approach is competence-oriented education, which aims at the complex of knowledge acquisition and methods of practical activity, through which a person successfully implements himself/herself in various fields of life. The most important specificity of the competency-based approach lies in the fact that not the "ready-made knowledge" transmitted by the teacher is assimilated, but "the conditions of the origin of this knowledge are traced".

Sign in / Sign up

Export Citation Format

Share Document