Rating Scale Data and its Utility: Additional Evidence to be Explored in the Medical Education Continuum

The purposes of this study are to assess the utility of self-appraisals and ratings of program directors (PDs) and to introduce new ways to use self-assessed and rating scale data. The data for this study was collected from graduates of our school who were enrolled in different residency programs around the country. The interns and PDs completed a similar set of questions. The correlation between the ratings of the two group was .21. The Cronbach’s alpha of interns and PD surveys were .89 and .97, respectively. The interns consistently rated themselves lower compared to ratings the PDs assigned them. The two groups agreed on the areas of strengths and weaknesses based on their mean ratings and rank-ordering of competencies. This study proposes that lowest mean ratings of measures that appear at the bottom in the rank-ordering be considered as areas that deserve special attention. The results of this study brought validity evidence to the utility of self-appraisals and PD’s ratings of interns.

Feedback Quality in an Aligned Teacher-Training Program

Background and Objectives: Residents as teachers (RAT) and medical students as teachers (MSAT) programs are important for the development of future physicians. In 2010, Northwestern University Feinberg School of Medicine (NUFSM) aligned RAT and MSAT programs, which created experiential learning opportunities in teaching and feedback across the graduate and undergraduate medical education continuum. The purpose of this study was to provide a curricular overview of the aligned program and to evaluate early outcomes through analysis of narrative feedback quality and participant satisfaction. Methods: Program evaluation occurred through analysis of written feedback quality provided within the aligned program and postparticipation satisfaction surveys. A total of 445 resident feedback narratives were collected from 2013 to 2016. We developed a quality coding scheme using an operational definition of feedback. After independent coding of feedback quality, an expert panel established coding consensus. We evaluated program satisfaction and perceived importance through posttraining surveys in residents and fourth-year medical students (M4s). Results: Seventy-nine residents participated in the aligned program and provided high-quality feedback with a relative quality rating of 2.71 (scale 0-3). Consistently high-quality written feedback was provided over the duration of the program and regardless of years of resident participation. Posttraining surveys demonstrated high levels of satisfaction and perceived importance of the program to both residents and M4s. Conclusions: The aligned RAT and MSAT program across the medical education continuum provided experiential learning opportunities for future physician educators with evidence of high-quality written feedback to learners and program satisfaction.

YouTube as an Educational Resource in Medical Education: A Scoping Review (Preprint)

BACKGROUND Video has been a powerful teaching and learning tool in medical education, enabling knowledge, skill and attitude formation in a variety of areas and reaching learners with various learning and communication styles. The millennial generation has grown up with video at their fingertips at any time and any place, and social networking sites such as YouTube enable the sharing of video amongst a vast online community. YouTube has emerged as a growing educational resource for both learners and medical educators. However, the usefulness of YouTube in supporting teaching and learning across the continuum of medical education has not been explored in detail. Given the increasing usage of YouTube in medical education, a review of the literature on YouTube and its utilization in medical education could inform more effective adoption and usage by institutions, educators, practitioners and learners. OBJECTIVE To explore the use of YouTube across the medical education continuum. METHODS A scoping review of the literature was performed. PubMed, ERIC and CINAHL were searched for literature on YouTube usage for medical education purposes published between 2005 and 2017 using a combination of search terms. Articles were screened using a defined set of inclusion criteria. Key items of information from each paper were collated using a data extraction tool and common themes from the literature were identified via thematic analysis. RESULTS Of the 113 articles initially identified in the literature search, 31 met inclusion criteria. Results show that a large number of YouTube videos exist across a variety of medical topic areas. However, only a small number of studies have evaluated the effectiveness of YouTube as an educational intervention and these are largely limited to learner satisfaction only. YouTube does offer the opportunity for educators to share videos and for learners to access a wide array of video materials. However, a majority of studies involving the assessment of content quality suggest that YouTube videos may be an inadequate source of information for learning due to the high variability of content. A key limitation in many of these studies was that searches were mainly restricted to publicly available video content. CONCLUSIONS There is a paucity of research and evaluative work surrounding the use of YouTube as an educational resource across the medical education continuum. The quality of the content on YouTube is highly variable due to a lack of peer-review. Further evaluation of the effective integration of YouTube in medical education would inform further understanding and future practice.