scholarly journals Evaluation of Surgical Skills during Robotic Surgery by Deep Learning-Based Multiple Surgical Instrument Tracking in Training and Actual Operations

2020 ◽  
Vol 9 (6) ◽  
pp. 1964
Author(s):  
Dongheon Lee ◽  
Hyeong Won Yu ◽  
Hyungju Kwon ◽  
Hyoun-Joong Kong ◽  
Kyu Eun Lee ◽  
...  
Keyword(s):  
Deep Learning ◽  
Robotic Surgery ◽  
Prediction Models ◽  
Pearson Correlation ◽  
Surgical Skills ◽  
Surgical Instruments ◽  
Skill Assessment ◽  
Surgical Instrument ◽  
Surgical Skill ◽  
Instrument Tracking

As the number of robotic surgery procedures has increased, so has the importance of evaluating surgical skills in these techniques. It is difficult, however, to automatically and quantitatively evaluate surgical skills during robotic surgery, as these skills are primarily associated with the movement of surgical instruments. This study proposes a deep learning-based surgical instrument tracking algorithm to evaluate surgeons’ skills in performing procedures by robotic surgery. This method overcame two main drawbacks: occlusion and maintenance of the identity of the surgical instruments. In addition, surgical skill prediction models were developed using motion metrics calculated from the motion of the instruments. The tracking method was applied to 54 video segments and evaluated by root mean squared error (RMSE), area under the curve (AUC), and Pearson correlation analysis. The RMSE was 3.52 mm, the AUC of 1 mm, 2 mm, and 5 mm were 0.7, 0.78, and 0.86, respectively, and Pearson’s correlation coefficients were 0.9 on the x-axis and 0.87 on the y-axis. The surgical skill prediction models showed an accuracy of 83% with Objective Structured Assessment of Technical Skill (OSATS) and Global Evaluative Assessment of Robotic Surgery (GEARS). The proposed method was able to track instruments during robotic surgery, suggesting that the current method of surgical skill assessment by surgeons can be replaced by the proposed automatic and quantitative evaluation method.

scholarly journals Automation of surgical skill assessment using a three-stage machine learning algorithm

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Joël L. Lavanchy ◽  
Joel Zindel ◽  
Kadir Kirtac ◽  
Isabell Twick ◽  
Enes Hosgor ◽  
...  
Keyword(s):  
Machine Learning ◽  
Learning Algorithm ◽  
Surgical Skills ◽  
Adverse Outcomes ◽  
Machine Learning Algorithms ◽  
Surgical Instruments ◽  
Skill Assessment ◽  
Surgical Skill ◽  
Surgical Skill Assessment ◽  
Motion Features

AbstractSurgical skills are associated with clinical outcomes. To improve surgical skills and thereby reduce adverse outcomes, continuous surgical training and feedback is required. Currently, assessment of surgical skills is a manual and time-consuming process which is prone to subjective interpretation. This study aims to automate surgical skill assessment in laparoscopic cholecystectomy videos using machine learning algorithms. To address this, a three-stage machine learning method is proposed: first, a Convolutional Neural Network was trained to identify and localize surgical instruments. Second, motion features were extracted from the detected instrument localizations throughout time. Third, a linear regression model was trained based on the extracted motion features to predict surgical skills. This three-stage modeling approach achieved an accuracy of 87 ± 0.2% in distinguishing good versus poor surgical skill. While the technique cannot reliably quantify the degree of surgical skill yet it represents an important advance towards automation of surgical skill assessment.

scholarly journals Deep neural networks for the assessment of surgical skills: A systematic review

2021 ◽  
pp. 154851292110345
Author(s):  
Erim Yanik ◽  
Xavier Intes ◽  
Uwe Kruger ◽  
Pingkun Yan ◽  
David Diller ◽  
...  
Keyword(s):  
Systematic Review ◽  
Neural Networks ◽  
Deep Neural Networks ◽  
Surgical Skills ◽  
Video Data ◽  
Skill Assessment ◽  
Assessment Process ◽  
Surgical Skill ◽  
Surgical Trainee ◽  
Surgical Skill Assessment

Surgical training in medical school residency programs has followed the apprenticeship model. The learning and assessment process is inherently subjective and time-consuming. Thus, there is a need for objective methods to assess surgical skills. Here, we use the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to systematically survey the literature on the use of Deep Neural Networks for automated and objective surgical skill assessment, with a focus on kinematic data as putative markers of surgical competency. There is considerable recent interest in deep neural networks (DNNs) due to the availability of powerful algorithms, multiple datasets, some of which are publicly available, as well as efficient computational hardware to train and host them. We have reviewed 530 papers, of which we selected 25 for this systematic review. Based on this review, we concluded that DNNs are potent tools for automated, objective surgical skill assessment using both kinematic and video data. The field would benefit from large, publicly available, annotated datasets representing the surgical trainee and expert demographics and multimodal data beyond kinematics and videos.

scholarly journals Surgical skill assessment using machine learning algorithms

2021 ◽  
Vol 108 (Supplement_4) ◽  
Author(s):  
J L Lavanchy ◽  
J Zindel ◽  
K Kirtac ◽  
I Twick ◽  
E Hosgor ◽  
...  
Keyword(s):  
Machine Learning ◽  
Clinical Outcomes ◽  
Learning Algorithm ◽  
Learning Algorithms ◽  
Movement Patterns ◽  
Surgical Skills ◽  
Machine Learning Algorithms ◽  
Skill Assessment ◽  
Surgical Skill ◽  
Surgical Skill Assessment

Abstract Objective Surgical skill is correlated with clinical outcomes. Therefore, the assessment of surgical skill is of major importance to improve clinical outcomes and increase patient safety. However, surgical skill assessment often lacks objectivity and reproducibility. Furthermore, it is time-consuming and expensive. Therefore, we developed an automated surgical skill assessment using machine learning algorithms. Methods Surgical skills were assessed in videos of laparoscopic cholecystectomy using a three-step machine learning algorithm. First, a three-dimensional convolutional neural network was trained to localize and classify the instruments within the videos. Second, movement patterns of the instruments were recorded over time and extracted. Third, the movement patterns were correlated with human surgical skill ratings using a linear regression model to predict surgical skill ratings automatically. Machine ratings were compared against human ratings of four board certified surgeons using a score ranging from 1 (poor skills) to 5 (excellent skills). Results Human raters and machine learning algorithms assessed surgical skills in 242 videos. Inter-rater reliability for human raters was excellent (79%, 95%CI 72-85%). Instrument detection showed an average precision of 78% and average recall of 82%. Machine learning algorithms showed an 87% accuracy in predicting good or poor surgical skills, when compared to human raters. Conclusion Machine learning algorithms can be trained to distinguish good and poor surgical skills with high accuracy. This work was published in Sci Rep 11, 5197 (2021). https://doi.org/10.1038/s41598-021-84295-6

scholarly journals Kinetic Measurement Platform for Open Surgical Skill Assessment

2017 ◽  
Author(s):  
Lasitha Wijayarathne ◽  
Frank L. Hammond
Keyword(s):  
Kinetic Data ◽  
Assessment System ◽  
Surgical Instruments ◽  
Skill Assessment ◽  
Surgical Skill ◽  
Data Set ◽  
Surgical Skill Assessment ◽  
Kinematic Data ◽  
Motion Trajectories ◽  
Tissue Interactions

Current surgical skill assessment methods are often based on the kinematics of manual surgical instruments during tool-tissue interactions. Though kinematic data are generally regarded as a sufficient basis for skill assessment, the inclusion of kinetic information would allow the assessment of measures such as “respect for tissue” and force control, which are also important aspects of surgical proficiency. Kinetic data would also provide a richer data set upon which automated surgical motion segmentation and classification algorithms can be developed. However, the kinetics of tool-tissue interactions are seldom included in assessments, due largely to the difficulty of mounting small sensors — typically silicon strain gauges — onto surgical instruments to capture force data. Electromagnetic (EM) or optical trackers used for kinematic measurement are often tethered, and thus having tethered force sensors also mounted on the same surgical instruments would complicate the experimental process and could affect/distort the acquired data by impeding the natural manual motions of surgeons. We present a surgical skill assessment platform which places the kinetic sensors in the environment, not on the instruments, to reduce the physical encumbrance of the system to the surgeon. This system can capture kinetic data using a standalone force/torque sensor embedded in a custom designed workspace platform, and kinematic data using EM trackers placed on the instruments. This portable platform enables the empirical characterization of open surgery motion trajectories and corresponding kinetic data without need for a centralized acquisition site, and will eventually be integrated into a completely untethered skill assessment system.

scholarly journals GPTransformer: A Transformer-Based Deep Learning Method for Predicting Fusarium Related Traits in Barley

2021 ◽  
Vol 12 ◽  
Author(s):  
Sheikh Jubair ◽  
James R. Tucker ◽  
Nathan Henderson ◽  
Colin W. Hiebert ◽  
Ana Badea ◽  
...  
Keyword(s):  
Machine Learning ◽  
Deep Learning ◽  
Molecular Markers ◽  
Fusarium Head Blight ◽  
High Throughput ◽  
Genomic Prediction ◽  
Prediction Models ◽  
Spring Barley ◽  
Pearson Correlation ◽  
Head Blight

Fusarium head blight (FHB) incited by Fusarium graminearum Schwabe is a devastating disease of barley and other cereal crops worldwide. Fusarium head blight is associated with trichothecene mycotoxins such as deoxynivalenol (DON), which contaminates grains, making them unfit for malting or animal feed industries. While genetically resistant cultivars offer the best economic and environmentally responsible means to mitigate disease, parent lines with adequate resistance are limited in barley. Resistance breeding based upon quantitative genetic gains has been slow to date, due to intensive labor requirements of disease nurseries. The production of a high-throughput genome-wide molecular marker assembly for barley permits use in development of genomic prediction models for traits of economic importance to this crop. A diverse panel consisting of 400 two-row spring barley lines was assembled to focus on Canadian barley breeding programs. The panel was evaluated for FHB and DON content in three environments and over 2 years. Moreover, it was genotyped using an Illumina Infinium High-Throughput Screening (HTS) iSelect custom beadchip array of single nucleotide polymorphic molecular markers (50 K SNP), where over 23 K molecular markers were polymorphic. Genomic prediction has been demonstrated to successfully reduce FHB and DON content in cereals using various statistical models. Herein, we have studied an alternative method based on machine learning and compare it with a statistical approach. The bi-allelic SNPs represented pairs of alleles and were encoded in two ways: as categorical (–1, 0, 1) or using Hardy-Weinberg probability frequencies. This was followed by selecting essential genomic markers for phenotype prediction. Subsequently, a Transformer-based deep learning algorithm was applied to predict FHB and DON. Apart from the Transformer method, a Residual Fully Connected Neural Network (RFCNN) was also applied. Pearson correlation coefficients were calculated to compare true vs. predicted outputs. Models which included all markers generally showed marginal improvement in prediction. Hardy-Weinberg encoding generally improved correlation for FHB (6.9%) and DON (9.6%) for the Transformer network. This study suggests the potential of the Transformer based method as an alternative to the popular BLUP model for genomic prediction of complex traits such as FHB or DON, having performed equally or better than existing machine learning and statistical methods.

Deep-Learning for Motion Compensation in Robotic Surgery

2017 ◽  
Author(s):  
P Triantafyllou ◽  
◽  
J Liu ◽  
G. Z Yang ◽  
S Giannarou
Keyword(s):  
Deep Learning ◽  
Robotic Surgery ◽  
Motion Compensation

Applying Deep Learning to Audit Procedures: An Illustrative Framework

2019 ◽  
Vol 33 (3) ◽  
pp. 89-109 ◽  
Author(s):  
Ting (Sophia) Sun
Keyword(s):  
Decision Making ◽  
Deep Learning ◽  
Data Warehouse ◽  
Visual Recognition ◽  
Prediction Models ◽  
Historical Data ◽  
Structured Data ◽  
Text Understanding ◽  
Audit Data ◽  
Learning Functions

SYNOPSIS This paper aims to promote the application of deep learning to audit procedures by illustrating how the capabilities of deep learning for text understanding, speech recognition, visual recognition, and structured data analysis fit into the audit environment. Based on these four capabilities, deep learning serves two major functions in supporting audit decision making: information identification and judgment support. The paper proposes a framework for applying these two deep learning functions to a variety of audit procedures in different audit phases. An audit data warehouse of historical data can be used to construct prediction models, providing suggested actions for various audit procedures. The data warehouse will be updated and enriched with new data instances through the application of deep learning and a human auditor's corrections. Finally, the paper discusses the challenges faced by the accounting profession, regulators, and educators when it comes to applying deep learning.

scholarly journals Exploiting Data Analytics and Deep Learning Systems to Support Pavement Maintenance Decisions

2021 ◽  
Vol 11 (6) ◽  
pp. 2458
Author(s):  
Ronald Roberts ◽  
Laura Inzerillo ◽  
Gaetano Di Mino
Keyword(s):  
Deep Learning ◽  
Management Practices ◽  
Prediction Models ◽  
Road Networks ◽  
Pavement Management ◽  
Efficient Operation ◽  
Pavement Maintenance ◽  
Goods And Services ◽  
Maintenance Decisions ◽  
Computational Systems

Road networks are critical infrastructures within any region and it is imperative to maintain their conditions for safe and effective movement of goods and services. Road Management, therefore, plays a key role to ensure consistent efficient operation. However, significant resources are required to perform necessary maintenance activities to achieve and maintain high levels of service. Pavement maintenance can typically be very expensive and decisions are needed concerning planning and prioritizing interventions. Data are key towards enabling adequate maintenance planning but in many instances, there is limited available information especially in small or under-resourced urban road authorities. This study develops a roadmap to help these authorities by using flexible data analysis and deep learning computational systems to highlight important factors within road networks, which are used to construct models that can help predict future intervention timelines. A case study in Palermo, Italy was successfully developed to demonstrate how the techniques could be applied to perform appropriate feature selection and prediction models based on limited data sources. The workflow provides a pathway towards more effective pavement maintenance management practices using techniques that can be readily adapted based on different environments. This takes another step towards automating these practices within the pavement management system.

scholarly journals A Novel Load Forecasting Approach Based on Smart Meter Data Using Advance Preprocessing and Hybrid Deep Learning

2021 ◽  
Vol 11 (6) ◽  
pp. 2742
Author(s):  
Fatih Ünal ◽  
Abdulaziz Almalaq ◽  
Sami Ekici
Keyword(s):  
Deep Learning ◽  
Energy Consumption ◽  
Prediction Models ◽  
Short Term Memory ◽  
Critical Role ◽  
Load Forecasting ◽  
Unified Framework ◽  
Short Term ◽  
Planning And Scheduling ◽  
Short Term Load Forecasting

Short-term load forecasting models play a critical role in distribution companies in making effective decisions in their planning and scheduling for production and load balancing. Unlike aggregated load forecasting at the distribution level or substations, forecasting load profiles of many end-users at the customer-level, thanks to smart meters, is a complicated problem due to the high variability and uncertainty of load consumptions as well as customer privacy issues. In terms of customers’ short-term load forecasting, these models include a high level of nonlinearity between input data and output predictions, demanding more robustness, higher prediction accuracy, and generalizability. In this paper, we develop an advanced preprocessing technique coupled with a hybrid sequential learning-based energy forecasting model that employs a convolution neural network (CNN) and bidirectional long short-term memory (BLSTM) within a unified framework for accurate energy consumption prediction. The energy consumption outliers and feature clustering are extracted at the advanced preprocessing stage. The novel hybrid deep learning approach based on data features coding and decoding is implemented in the prediction stage. The proposed approach is tested and validated using real-world datasets in Turkey, and the results outperformed the traditional prediction models compared in this paper.

Sign in / Sign up

Export Citation Format

Share Document