scholarly journals Exploring Eye‐tracking Technology as an Assessment Tool for Point‐of‐care Ultrasound Training

2020 ◽  
Author(s):  
Wei Feng Lee ◽  
Jordan Chenkin
Keyword(s):  
Eye Tracking ◽  
Assessment Tool ◽  
Point Of Care ◽  
Point Of Care Ultrasound ◽  
Tracking Technology

scholarly journals LO85: Development of a competency based assessment tool for emergency department point of care ultrasound

CJEM ◽  
2019 ◽  
Vol 21 (S1) ◽  
pp. S38-S39
Author(s):  
C. McKaigney ◽  
C. Bell ◽  
A. Hall
Keyword(s):  
Delphi Method ◽  
Assessment Tool ◽  
Point Of Care ◽  
Image Acquisition ◽  
Clinical Indication ◽  
Point Of Care Ultrasound ◽  
Clinical Integration ◽  
Modified Delphi Method ◽  
Modified Delphi ◽  
Image Optimization

Innovation Concept: Assessment of residents' Point of Care Ultrasound (PoCUS) competency currently relies on heterogenous and unvalidated methods, such as the completion of a number of proctored studies. Although number of performed studies may be associated with ability, it is not necessarily a surrogate for competence. Our goal was to create a single Ultrasound Competency Assessment Tool (UCAT) using domain-anchored entrustment scoring. Methods: The UCAT was developed as an anchored global assessment score, building on a previously validated simulation-based assessment tool. It was designed to measure performance across the domains of Preparation, Image Acquisition, Image Optimization, and Clinical Integration, in addition to providing a final entrustment score (i.e., OSCORE). A modified Delphi method was used to establish national expert consensus on anchors for each domain. Three surveys were distributed to the CAEP Ultrasound Committee between July-November 2018. The first survey asked members to appraise and modify a list of anchor options created by the authors. Next, collated responses from the first survey were redistributed for a re-appraisal. Finally, anchors obtaining >65% approval from the second survey were condensed and redistributed for final consensus. Curriculum, Tool or Material: Twenty-two, 26, and 22 members responded to the surveys, respectively. Each anchor achieved >90% final agreement. The final anchors for the domains were: Preparation – positioning, initial settings, ensures clean transducer, probe selection, appropriate clinical indication; Image Acquisition – appropriate measurements, hand position, identifies landmarks, visualization of target, efficiency of probe motion, troubleshoots technical limitations; Image Optimization – centers area of interest, overall image quality, troubleshoots patient obstacles, optimizes settings; Clinical Integration – appropriate interpretation, understands limitations, utilizes information appropriately, performs multiple scans if needed, communicates findings, considers false positive and negative causes of findings. Conclusion: The UCAT is a novel assessment tool that has the potential to play a central role in the training and evaluation of residents. Our use of a modified Delphi method, involving key stakeholders in PoCUS education, ensures that the UCAT has a high degree of process and content validity. An important next step in determining its construct validity is to evaluate the use of the UCAT in a multi-centered examination setting.

scholarly journals Development and Validation of an Assessment Tool for Competency in Critical Care Ultrasound

2015 ◽  
Vol 7 (4) ◽  
pp. 567-573 ◽  
Author(s):  
Paru Patrawalla ◽  
Lewis Ari Eisen ◽  
Ariel Shiloh ◽  
Brijen J. Shah ◽  
Oleksandr Savenkov ◽  
...  
Keyword(s):  
Critical Care ◽  
Delphi Method ◽  
Assessment Tool ◽  
Point Of Care ◽  
Critical Care Medicine ◽  
Competency Assessment ◽  
Point Of Care Ultrasound ◽  
Modified Delphi Method ◽  
Modified Delphi ◽  
Critical Care Ultrasound

ABSTRACT Background Point-of-care ultrasound is an emerging technology in critical care medicine. Despite requirements for critical care medicine fellowship programs to demonstrate knowledge and competency in point-of-care ultrasound, tools to guide competency-based training are lacking. Objective We describe the development and validity arguments of a competency assessment tool for critical care ultrasound. Methods A modified Delphi method was used to develop behaviorally anchored checklists for 2 ultrasound applications: “Perform deep venous thrombosis study (DVT)” and “Qualify left ventricular function using parasternal long axis and parasternal short axis views (Echo).” One live rater and 1 video rater evaluated performance of 28 fellows. A second video rater evaluated a subset of 10 fellows. Validity evidence for content, response process, and internal consistency was assessed. Results An expert panel finalized checklists after 2 rounds of a modified Delphi method. The DVT checklist consisted of 13 items, including 1.00 global rating step (GRS). The Echo checklist consisted of 14 items, and included 1.00 GRS for each of 2 views. Interrater reliability evaluated with a Cohen kappa between the live and video rater was 1.00 for the DVT GRS, 0.44 for the PSLA GRS, and 0.58 for the PSSA GRS. Cronbach α was 0.85 for DVT and 0.92 for Echo. Conclusions The findings offer preliminary evidence for the validity of competency assessment tools for 2 applications of critical care ultrasound and data on live versus video raters.

scholarly journals LO82: Exploring eye-tracking technology to assess competency in point-of-care ultrasound

CJEM ◽  
2020 ◽  
Vol 22 (S1) ◽  
pp. S37-S37
Author(s):  
W. Lee ◽  
J. Chenkin
Keyword(s):  
Eye Tracking ◽  
Visual Processing ◽  
Average Duration ◽  
Point Of Care ◽  
Secondary Outcome ◽  
Great Promise ◽  
Point Of Care Ultrasound ◽  
The Gaze ◽  
Area Of Interest ◽  
Gaze Patterns

Introduction: Assessment of point-of-care ultrasound (POCUS) competency has been reliant on practical, visual and written examinations performed 1-on-1 with an examiner. These tools attempt to assess competency through subjective ratings, checklists and multiple-choice questions that are labour intensive using surrogate measures. Eye-tracking has been used on a limited basis in various fields of medicine for training and assessment. This technology explores visual processing and holds great promise as a tool to monitor training progress towards the development of expertise. We hypothesize that eye-tracking may differentiate novices and experts as they progress to become competent in interpretation of POCUS images and provide an objective measure in assessment of competency. Methods: Medical students, residents and attending physicians working in an academic emergency department were recruited. Participants viewed a series of 16 ultrasound video clips in a POCUS protocol for Focused Assessment using Sonography in Trauma (FAST). The gaze pattern of the participants was recorded using a commercially available eye-tracking device. The primary outcome was the gaze parameters including total gaze time in the area of interest (AOI), average time to fixation on the AOI, number of fixations in the AOI and average duration of first fixation on the AOI. Secondary outcome was the accuracy on the interpretation of the FAST scan. Results: Four novices and eight experts completed this study. The total gaze time in the AOI (mean +/- SD) was 76.72 +/- 18.84s among experts vs 53.64 +/- 10.33s among novices (p = 0.048), average time to fixation on the AOI was 0.561 +/- 0.319s vs 1.048 +/- 0.280s (p = 0.027), number of fixations in the AOI was 158.9 +/- 29.0 vs 121.8 +/- 17.5 (p = 0.042) and average duration of first fixation was 0.444 +/- 0.119s vs 0.390 +/- 0.024s (p = 0.402). The accuracy of the answers was 79.7 +/- 14.1% vs 45.3 +/- 21.9% (p = 0.007). Conclusion: In this pilot study, eye tracking shows potential to differentiate between POCUS experts and novices by their gaze patterns. Gaze patterns captured by eye tracking may not necessarily translate to cognitive processing. However, it allows educators to visualise the thought processes of the learner by their gaze patterns and provide insight on how to guide them towards competency. Future studies are needed to further validate the metrics for competency in POCUS applications.

The Use of Eye Tracking Technology in Aesthetic Surgery: Analyzing Changes in Facial Attention Following Surgery

2020 ◽  
Vol 40 (12) ◽  
pp. 1269-1279
Author(s):  
Russell S Frautschi ◽  
Nadeera Dawlagala ◽  
Eric W Klingemier ◽  
Hannah S England ◽  
Nicholas R Sinclair ◽  
...  
Keyword(s):  
Eye Tracking ◽  
Quantitative Data ◽  
Dwell Time ◽  
Assessment Tool ◽  
Aesthetic Surgery ◽  
Facial Rejuvenation ◽  
Fixation Time ◽  
Postoperative Changes ◽  
Areas Of Interest ◽  
Tracking Technology

Abstract Background The ability to quantitatively analyze how we look at a face and determine if this changes following facial surgery should be of interest to the plastic surgeon. Eye tracking technology (ETT) provides the ability to record where observers fixate when viewing a facial image, enabling quantitative data to be obtained comparing pre- and postoperative changes. Objectives The authors sought to investigate ETT as a novel outcome assessment tool, determining if facial rejuvenation surgery shifts attention away from the prominent signs of aging, and if so, where this attention shifts. Methods Twenty-five volunteers viewed 32 randomized frontal, oblique, and lateral images of 11 patients pre- and post-facelift. An eye movement monitoring system recorded the observer’s eye position, net dwell time, fixation count, fixation time, and revisits into predefined areas of interest. Data were grouped and analyzed by angle and areas of interest. Paired t tests were employed to detect significant differences in pre- and post-images. Results On frontal images, less dwell time, fixations, and revisits were noted on the bottom third, forehead, perioral region, and neck (P < 0.05). On the lateral view, less visual attention was given to the neck, upper third, and perioral region, with more time in the cheek, nose, and middle third (P < 0.05). On oblique images, less attention was given to the neck and upper lid with more aimed at the middle third of the face (P < 0.05). Conclusions ETT provides quantitative data post-facial rejuvenation. Facial aesthetic surgery does alter where observers look when viewing a face, decreasing the time spent inspecting the prominent signs of aging.

scholarly journals Consensus-Based Expert Development of Critical Items for Direct Observation of Point-of-Care Ultrasound Skills

2020 ◽  
Vol 12 (2) ◽  
pp. 176-184 ◽  
Author(s):  
Irene W. Y. Ma ◽  
Janeve Desy ◽  
Michael Y. Woo ◽  
Andrew W. Kirkpatrick ◽  
Vicki E. Noble
Keyword(s):  
Assessment Tool ◽  
Point Of Care ◽  
Point Of Care Ultrasound ◽  
Cut Scores ◽  
Medical Specialties ◽  
Modified Delphi ◽  
Cut Score ◽  
Modified Delphi Technique ◽  
Workplace Based Assessment ◽  
Competency Based

ABSTRACT Background Point-of-care ultrasound (POCUS) is increasingly used in a number of medical specialties. To support competency-based POCUS education, workplace-based assessments are essential. Objective We developed a consensus-based assessment tool for POCUS skills and determined which items are critical for competence. We then performed standards setting to set cut scores for the tool. Methods Using a modified Delphi technique, 25 experts voted on 32 items over 3 rounds between August and December 2016. Consensus was defined as agreement by at least 80% of the experts. Twelve experts then performed 3 rounds of a standards setting procedure in March 2017 to establish cut scores. Results Experts reached consensus for 31 items to include in the tool. Experts reached consensus that 16 of those items were critically important. A final cut score for the tool was established at 65.2% (SD 17.0%). Cut scores for critical items are significantly higher than those for noncritical items (76.5% ± SD 12.4% versus 53.1% ± SD 12.2%, P < .0001). Conclusions We reached consensus on a 31-item workplace-based assessment tool for identifying competence in POCUS. Of those items, 16 were considered critically important. Their importance is further supported by higher cut scores compared with noncritical items.

scholarly journals 174 Use of Hand Motion Analysis as Assessment Tool for Cardiac and Lung Point-of-Care Ultrasound

2018 ◽  
Vol 72 (4) ◽  
pp. S71-S72 ◽  
Author(s):  
D.J. Ackil ◽  
A. Toney ◽  
R. Good ◽  
D. Ross ◽  
R. Germano ◽  
...  
Keyword(s):  
Motion Analysis ◽  
Assessment Tool ◽  
Point Of Care ◽  
Point Of Care Ultrasound ◽  
Hand Motion ◽  
Lung Point

Evaluation of a survey for acute care programme directors on the utilisation of point-of-care ultrasound

2021 ◽  
pp. postgradmedj-2021-140127
Author(s):  
Annie Wang ◽  
Melissa McCabe ◽  
Esther Gow-Lee ◽  
Stephanie James ◽  
Briahnna Austin ◽  
...  
Keyword(s):  
Emergency Medicine ◽  
Acute Care ◽  
Assessment Tool ◽  
Point Of Care ◽  
Educational Opportunities ◽  
Point Of Care Ultrasound ◽  
Diagnostic Modality ◽  
Care Programme ◽  
Institutional Review ◽  
Primary Assessment

PurposePoint-of-care ultrasound (POCUS) is ultrasound brought to the patient’s bedside and performed in ‘real time’ by the healthcare provider. The utility of POCUS to facilitate management of the acutely ill patient has been demonstrated for multiple pathologies. However, the integration of ultrasonography and echocardiography training into residency curriculum varies across the acute care specialties.Study designAfter an institutional review board approval, anaesthesiology, emergency medicine, family medicine, internal medicine, paediatrics and general surgery programme directors (PDs) were surveyed. The survey consisted of 11 questions evaluating the primary bedside assessment tool for common acute care situations, POCUS topics that the PDs were comfortable practising and topics that the PDs felt were useful for their specialty. Barriers to POCUS use, certification and documentation were also surveyed.ResultsOverall, 270 PD surveys were completed. The preferred primary assessment tool for common acute care situations varied with specialty; emergency medicine PDs consistently responded that POCUS was the diagnostic modality of choice (p<0.0001). The majority of the PDs reported lack of educational opportunities as the primary barrier to learning POCUS (64%). Most PDs indicated that POCUS examinations should be documented (95.7%), and 39% reported that departmental certification would be sufficient.ConclusionsThis study is the first to evaluate differences in the preferred initial bedside assessment tool between the acute care specialties. Although POCUS is a superior tool for evaluating acute pathologies, disconnect between education and utilisation remains. This study highlights the need to incorporate POCUS into the acute care specialty curriculum.

Sign in / Sign up

Export Citation Format

Share Document