Multisensor fusion in gastroenterology domain through video and echo endoscopic image combination: a challenge

2001 ◽  
Author(s):  
Renaud Debon ◽  
Clara Le Guillou ◽  
Jean-Michel Cauvin ◽  
Basel Solaiman ◽  
Christian Roux
Keyword(s):  
Multisensor Fusion ◽  
Endoscopic Image

Multisensor Fusion Processing for Enhanced Radar Imaging

2000 ◽  
Author(s):  
Laura D. Vann ◽  
Kevin M. Cuomo ◽  
Jean E. Piou ◽  
Joseph T. Mayhan
Keyword(s):  
Radar Imaging ◽  
Multisensor Fusion

scholarly journals TOWARDS AN UNIFIED FRAMEWORK FOR VALIDATING DEEP LEARNING METHODS FOR COLORECTAL POLYPS: FIRST STEPS

2021 ◽  
Vol 108 (Supplement_3) ◽  
Author(s):  
L F Sánchez Peralta ◽  
J F Ortega Morán ◽  
Cr L Saratxaga ◽  
J B Pagador ◽  
A Picón ◽  
...  
Keyword(s):  
Deep Learning ◽  
Colorectal Polyps ◽  
Polyp Detection ◽  
Unified Framework ◽  
Endoscopic Image ◽  
Learning Techniques ◽  
Great Utility ◽  
Adenoma Detection ◽  
Definition Of ◽  
Basic Features

Abstract INTRODUCTION Deep learning techniques have significantly contributed to the field of medical imaging analysis. In case of colorectal cancer, they have shown a great utility for increasing the adenoma detection rate at colonoscopy, but a common validation methodology is still missing. In this study, we present preliminary efforts towards the definition of a validation framework. MATERIAL AND METHODS Different models based on different backbones and encoder-decoder architectures have been trained with a publicly available dataset that contains white light and NBI colonoscopy videos, with 76 different lesions from colonoscopy procedures in 48 human patients. A computer aided detection (CADe) demonstrator has been implemented to show the performance of the models. RESULTS This CADe demonstrator shows the areas detected as polyp by overlapping the predicted mask on the endoscopic image. It allows selecting the video to be used, among those from the test set. Although it only present basic features such as play, pause and moving to the next video, it easily loads the model and allows for visualization of results. The demonstrator is accompanied by a set of metrics to be used depending on the aimed task: polyp detection, localization and segmentation. CONCLUSIONS The use of this CADe demonstrator, together with a publicly available dataset and predefined metrics will allow for an easier and more fair comparison of methods. Further work is still required to validate the proposed framework.

Multimodal Sensor Fusion for Indoor Occupancy Determination

2015 ◽  
Vol 764-765 ◽  
pp. 1319-1323
Author(s):  
Rong Shue Hsiao ◽  
Ding Bing Lin ◽  
Hsin Piao Lin ◽  
Jin Wang Zhou
Keyword(s):  
Bayesian Networks ◽  
Sensor Fusion ◽  
Dynamic Bayesian Networks ◽  
Sensor Nodes ◽  
Detection Accuracy ◽  
Multisensor Fusion ◽  
Fusion Algorithm ◽  
Improve Energy Efficiency ◽  
Occupancy Detection ◽  
Automation Systems

Pyroelectric infrared (PIR) sensors can detect the presence of human without the need to carry any device, which are widely used for human presence detection in home/office automation systems in order to improve energy efficiency. However, PIR detection is based on the movement of occupants. For occupancy detection, PIR sensors have inherent limitation when occupants remain relatively still. Multisensor fusion technology takes advantage of redundant, complementary, or more timely information from different modal sensors, which is considered an effective approach for solving the uncertainty and unreliability problems of sensing. In this paper, we proposed a simple multimodal sensor fusion algorithm, which is very suitable to be manipulated by the sensor nodes of wireless sensor networks. The inference algorithm was evaluated for the sensor detection accuracy and compared to the multisensor fusion using dynamic Bayesian networks. The experimental results showed that a detection accuracy of 97% in room occupancy can be achieved. The accuracy of occupancy detection is very close to that of the dynamic Bayesian networks.

scholarly journals Real-time deep learning semantic segmentation during intra-operative surgery for 3D augmented reality assistance

2021 ◽  
Author(s):  
Leonardo Tanzi ◽  
Pietro Piazzolla ◽  
Francesco Porpiglia ◽  
Enrico Vezzetti
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Augmented Reality ◽  
Ad Hoc ◽  
Geodesic Distance ◽  
Semantic Segmentation ◽  
Endoscopic Image ◽  
Operative Surgery ◽  
Endoscopic Videos

Abstract Purpose The current study aimed to propose a Deep Learning (DL) and Augmented Reality (AR) based solution for a in-vivo robot-assisted radical prostatectomy (RARP), to improve the precision of a published work from our group. We implemented a two-steps automatic system to align a 3D virtual ad-hoc model of a patient’s organ with its 2D endoscopic image, to assist surgeons during the procedure. Methods This approach was carried out using a Convolutional Neural Network (CNN) based structure for semantic segmentation and a subsequent elaboration of the obtained output, which produced the needed parameters for attaching the 3D model. We used a dataset obtained from 5 endoscopic videos (A, B, C, D, E), selected and tagged by our team’s specialists. We then evaluated the most performing couple of segmentation architecture and neural network and tested the overlay performances. Results U-Net stood out as the most effecting architectures for segmentation. ResNet and MobileNet obtained similar Intersection over Unit (IoU) results but MobileNet was able to elaborate almost twice operations per seconds. This segmentation technique outperformed the results from the former work, obtaining an average IoU for the catheter of 0.894 (σ = 0.076) compared to 0.339 (σ = 0.195). This modifications lead to an improvement also in the 3D overlay performances, in particular in the Euclidean Distance between the predicted and actual model’s anchor point, from 12.569 (σ= 4.456) to 4.160 (σ = 1.448) and in the Geodesic Distance between the predicted and actual model’s rotations, from 0.266 (σ = 0.131) to 0.169 (σ = 0.073). Conclusion This work is a further step through the adoption of DL and AR in the surgery domain. In future works, we will overcome the limits of this approach and finally improve every step of the surgical procedure.

Sign in / Sign up

Export Citation Format

Share Document