EP.TH.589Comparison of Surgeon Gaze Behaviour Against Objective Skill Assessment in Open Inguinal Hernia Repair – A Pilot Study

Aims Technical skill is associated with improved postoperative outcomes. Adoption of a formalised high-stakes assessment of surgical skill is technically challenging and limited by the financial and human resources available. We aimed to assess the ability to adopt gaze behaviour analysis as an assessment of surgical skill within live open inguinal herniorrhaphy. Methods Surgeons’ gaze was measured with Tobii Pro eye-tracking Glasses 2 (Tobii AB). All grades of surgeons were included. Primary outcomes were dwell time (%) and fixation frequency (count/s), as markers of cognition, on areas of interest correlated to mean Objective Skill Assessment of Technical Skill score. Secondary outcomes assessed effort and concentration levels through maximum pupil diameter (mm) and rate of pupil change (mm/s) correlated to perceived workload (SURG-TLX). Three operative segments underwent analysis: mesh preparation, fixation and muscle closure. Spearman’s and Pearson’s correlation were performed with significance set at p < 0.05. Results 5 cases were analysed, totalling 270 minutes of video footage. All participants were senior surgical trainees and right-hand-dominant. The median number of hernia operations performed was 160 (range:100-500). The median ASA score of each patient participant was 2 (range:1-2). The median operation length was 45 mins (range:40-90 mins). There were no statistically significant primary outcomes from this pilot data (p > 0.05). Conclusions This pilot study demonstrated the feasibility of recording gaze behaviours for comparison against formal skills assessment to determine the role of eye tracking in live high stakes technical skills assessment. A full study will now commence based on formal power calculation.

What is Seen Through the Looking Glass: The Impact of Training on Practitioner Self-Rating of Motivational Interviewing Skills

Training efforts for evidenced based treatments require evaluation, yet the value of practitioner self-reports of skills acquisition has been questioned. Thus, a key issue concerns how accurately practitioners assess their own clinical skills. In the current study, 23 community practitioners participated in training of Motivational Interviewing (MI), completed standardized patient (SP) interviews before and after training, and provided self-ratings of MI elements after each interview. Interview recordings were later coded independently. Results suggest training contributed to: 1) reasonable agreement between practitioner and independent ratings; and 2) more effective use of MI, despite a tendency for practitioners to underestimate training gains. This micro-analysis of training documents initial skill gains along with increased practitioner self-awareness. Further, it exemplifies how practitioner self-ratings and objective skill assessment methods may be used in tandem to more fully describe practitioner learning.