Deep learning supported disease detection with multi-modality image fusion

2021 ◽  
pp. 1-24
Author(s):  
F. Sangeetha Francelin Vinnarasi ◽  
Jesline Daniel ◽  
J.T. Anita Rose ◽  
R. Pugalenthi
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Image Fusion ◽  
Clustering Algorithm ◽  
Medical Images ◽  
Disease Diagnosis ◽  
High Energy ◽  
Fusion Algorithm ◽  
Fused Image ◽  
Fuzzy C Means Clustering

Multi-modal image fusion techniques aid the medical experts in better disease diagnosis by providing adequate complementary information from multi-modal medical images. These techniques enhance the effectiveness of medical disorder analysis and classification of results. This study aims at proposing a novel technique using deep learning for the fusion of multi-modal medical images. The modified 2D Adaptive Bilateral Filters (M-2D-ABF) algorithm is used in the image pre-processing for filtering various types of noises. The contrast and brightness are improved by applying the proposed Energy-based CLAHE algorithm in order to preserve the high energy regions of the multimodal images. Images from two different modalities are first registered using mutual information and then registered images are fused to form a single image. In the proposed fusion scheme, images are fused using Siamese Neural Network and Entropy (SNNE)-based image fusion algorithm. Particularly, the medical images are fused by using Siamese convolutional neural network structure and the entropy of the images. Fusion is done on the basis of score of the SoftMax layer and the entropy of the image. The fused image is segmented using Fast Fuzzy C Means Clustering Algorithm (FFCMC) and Otsu Thresholding. Finally, various features are extracted from the segmented regions. Using the extracted features, classification is done using Logistic Regression classifier. Evaluation is performed using publicly available benchmark dataset. Experimental results using various pairs of multi-modal medical images reveal that the proposed multi-modal image fusion and classification techniques compete the existing state-of-the-art techniques reported in the literature.

scholarly journals ICONet: A Lightweight Network with Greater Environmental Adaptivity

Symmetry ◽  
2020 ◽  
Vol 12 (12) ◽  
pp. 2119
Author(s):  
Wei He ◽  
Yanmei Huang ◽  
Zanhao Fu ◽  
Yingcheng Lin
Keyword(s):  
Neural Network ◽  
Artificial Intelligence ◽  
Deep Learning ◽  
Time Complexity ◽  
Clustering Algorithm ◽  
Complex Environments ◽  
Illumination Condition ◽  
Fuzzy C Means Clustering ◽  
Fatigue Driving ◽  
Reduced Time

With the increasing popularity of artificial intelligence, deep learning has been applied to various fields, especially in computer vision. Since artificial intelligence is migrating from cloud to edge, deep learning nowadays should be edge-oriented and adaptive to complex environments. Aiming at these goals, this paper proposes an ICONet (illumination condition optimized network). Based on OTSU segmentation algorithm and fuzzy c-means clustering algorithm, the illumination condition classification subnet increases the environmental adaptivity of our network. The reduced time complexity and optimized size of our convolutional neural network (CNN) model enables the implementation of ICONet on edge devices. In the field of fatigue driving, we test the performance of ICONet on YawDD and self-collected datasets. Our network achieves a general accuracy of 98.56% and our models are about 590 kilobytes. Compared to other proposed networks, the ICONet shows significant success and superiority. Applying ICONet to fatigue driving detection is helpful to solve the symmetry of the needs of edge-oriented detection under complex illumination condition environments and the scarcity of related approaches.

scholarly journals Multi-Modality Medical Image Fusion Using Convolutional Neural Network and Contrast Pyramid

Sensors ◽  
2020 ◽  
Vol 20 (8) ◽  
pp. 2169 ◽  
Author(s):  
Kunpeng Wang ◽  
Mingyao Zheng ◽  
Hongyan Wei ◽  
Guanqiu Qi ◽  
Yuanyuan Li
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Image Fusion ◽  
Medical Image ◽  
Source Image ◽  
Fusion Algorithm ◽  
Convolutional Network ◽  
Structure Information ◽  
Medical Image Fusion ◽  
Fused Image

Medical image fusion techniques can fuse medical images from different morphologies to make the medical diagnosis more reliable and accurate, which play an increasingly important role in many clinical applications. To obtain a fused image with high visual quality and clear structure details, this paper proposes a convolutional neural network (CNN) based medical image fusion algorithm. The proposed algorithm uses the trained Siamese convolutional network to fuse the pixel activity information of source images to realize the generation of weight map. Meanwhile, a contrast pyramid is implemented to decompose the source image. According to different spatial frequency bands and a weighted fusion operator, source images are integrated. The results of comparative experiments show that the proposed fusion algorithm can effectively preserve the detailed structure information of source images and achieve good human visual effects.

scholarly journals Fast Multi-Focus Fusion Based on Deep Learning for Early-Stage Embryo Image Enhancement

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 863
Author(s):  
Vidas Raudonis ◽  
Agne Paulauskaite-Taraseviciene ◽  
Kristina Sutiene
Keyword(s):  
Deep Learning ◽  
Image Fusion ◽  
Early Stage ◽  
Image Data ◽  
Cell Detection ◽  
Processing Times ◽  
Fused Image ◽  
Stage Embryo ◽  
Early Stage Embryo

Background: Cell detection and counting is of essential importance in evaluating the quality of early-stage embryo. Full automation of this process remains a challenging task due to different cell size, shape, the presence of incomplete cell boundaries, partially or fully overlapping cells. Moreover, the algorithm to be developed should process a large number of image data of different quality in a reasonable amount of time. Methods: Multi-focus image fusion approach based on deep learning U-Net architecture is proposed in the paper, which allows reducing the amount of data up to 7 times without losing spectral information required for embryo enhancement in the microscopic image. Results: The experiment includes the visual and quantitative analysis by estimating the image similarity metrics and processing times, which is compared to the results achieved by two wellknown techniques—Inverse Laplacian Pyramid Transform and Enhanced Correlation Coefficient Maximization. Conclusion: Comparatively, the image fusion time is substantially improved for different image resolutions, whilst ensuring the high quality of the fused image.

scholarly journals An innovative image fusion algorithm based on wavelet transform and discrete fast curvelet transform

2011 ◽  
Vol 1 (3) ◽  
Author(s):  
T. Sumathi ◽  
M. Hemalatha
Keyword(s):  
Wavelet Transform ◽  
Image Fusion ◽  
Curvelet Transform ◽  
Relevant Information ◽  
Good Time ◽  
Discrete Wavelet ◽  
Fusion Algorithm ◽  
Time Frequency ◽  
Fused Image

AbstractImage fusion is the method of combining relevant information from two or more images into a single image resulting in an image that is more informative than the initial inputs. Methods for fusion include discrete wavelet transform, Laplacian pyramid based transform, curvelet based transform etc. These methods demonstrate the best performance in spatial and spectral quality of the fused image compared to other spatial methods of fusion. In particular, wavelet transform has good time-frequency characteristics. However, this characteristic cannot be extended easily to two or more dimensions with separable wavelet experiencing limited directivity when spanning a one-dimensional wavelet. This paper introduces the second generation curvelet transform and uses it to fuse images together. This method is compared against the others previously described to show that useful information can be extracted from source and fused images resulting in the production of fused images which offer clear, detailed information.

Computed tomography image denoising based on multi dose CT image fusion and extended sparse techniques

2021 ◽  
Author(s):  
Anuyogam Venkataraman
Keyword(s):  
Computed Tomography ◽  
Image Fusion ◽  
Region Of Interest ◽  
Source Image ◽  
High Dose ◽  
Fusion Algorithm ◽  
Quantitative Metrics ◽  
Fused Image ◽  
Fusion Methods ◽  
Computed Tomography Image

With the increasing utilization of X-ray Computed Tomography (CT) in medical diagnosis, obtaining higher quality image with lower exposure to radiation is a highly challenging task in image processing. Sparse representation based image fusion is one of the sought after fusion techniques among the current researchers. A novel image fusion algorithm based on focused vector detection is proposed in this thesis. Firstly, the initial fused vector is acquired by combining common and innovative sparse components of multi-dosage ensemble using Joint Sparse PCA fusion method utilizing an overcomplete dictionary trained using high dose images of the same region of interest from different patients. And then, the strongly focused vector is obtained by determining the pixels of low dose and medium dose vectors which have high similarity with the pixels of the initial fused vector using certain quantitative metrics. Final fused image is obtained by denoising and simultaneously integrating the strongly focused vector, initial fused vector and source image vectors in joint sparse domain thereby preserving the edges and other critical information needed for diagnosis. This thesis demonstrates the effectiveness of the proposed algorithms when experimented on different images and the qualitative and quantitative results are compared with some of the widely used image fusion methods.

scholarly journals Deep Learning for Caries Detection and Classification

Diagnostics ◽  
2021 ◽  
Vol 11 (9) ◽  
pp. 1672
Author(s):  
Luya Lian ◽  
Tianer Zhu ◽  
Fudong Zhu ◽  
Haihua Zhu
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Disease Diagnosis ◽  
Validation Dataset ◽  
Reference Dataset ◽  
Dice Coefficient ◽  
Learning Methods ◽  
Test Dataset ◽  
The Impact ◽  
Caries Lesions

Objectives: Deep learning methods have achieved impressive diagnostic performance in the field of radiology. The current study aimed to use deep learning methods to detect caries lesions, classify different radiographic extensions on panoramic films, and compare the classification results with those of expert dentists. Methods: A total of 1160 dental panoramic films were evaluated by three expert dentists. All caries lesions in the films were marked with circles, whose combination was defined as the reference dataset. A training and validation dataset (1071) and a test dataset (89) were then established from the reference dataset. A convolutional neural network, called nnU-Net, was applied to detect caries lesions, and DenseNet121 was applied to classify the lesions according to their depths (dentin lesions in the outer, middle, or inner third D1/2/3 of dentin). The performance of the test dataset in the trained nnU-Net and DenseNet121 models was compared with the results of six expert dentists in terms of the intersection over union (IoU), Dice coefficient, accuracy, precision, recall, negative predictive value (NPV), and F1-score metrics. Results: nnU-Net yielded caries lesion segmentation IoU and Dice coefficient values of 0.785 and 0.663, respectively, and the accuracy and recall rate of nnU-Net were 0.986 and 0.821, respectively. The results of the expert dentists and the neural network were shown to be no different in terms of accuracy, precision, recall, NPV, and F1-score. For caries depth classification, DenseNet121 showed an overall accuracy of 0.957 for D1 lesions, 0.832 for D2 lesions, and 0.863 for D3 lesions. The recall results of the D1/D2/D3 lesions were 0.765, 0.652, and 0.918, respectively. All metric values, including accuracy, precision, recall, NPV, and F1-score values, were proven to be no different from those of the experienced dentists. Conclusion: In detecting and classifying caries lesions on dental panoramic radiographs, the performance of deep learning methods was similar to that of expert dentists. The impact of applying these well-trained neural networks for disease diagnosis and treatment decision making should be explored.

scholarly journals An Efficient PET-MRI Medical Image Fusion based on IHS-NSCT-PCA Integrated Method

2019 ◽  
Vol 9 (2) ◽  
pp. 2073-2079
Keyword(s):  
Structural Data ◽  
Fusion Rule ◽  
Integrated Approach ◽  
Disease Diagnosis ◽  
Weighted Averaging ◽  
Fusion Algorithm ◽  
Integrated Method ◽  
Histogram Matching ◽  
Multiple Imaging ◽  
Fused Image

Merging of multiple imaging modalities leads to a single image that acquire high information content. These find useful applications in disease diagnosis and treatment planning. IHS-PCA method is a spatial domain approach for fusion that offersfinestvisibility but demands vast memory and it lacks steering information. We propose an integrated approach that incorporates NSCT combined with PCA utilizing IHS space and histogram matching. The fusion algorithm is applied on MRI with PET image and improved functional property was obtained. The IHS transform is a sharpening technique that converts multispectral image from RGB channels to Intensity Hue and Saturation independent values. Histogram matching is performed with intensity values of the two input images. Pathological details in images can be emphasized in multi-scale and multi-directions by using PCA withNSCT. Fusion rule applied is weighted averaging andprincipal components are used for dimensionality reduction. Inverse NSCT and Inverse IHS are performed so as to obtain the fused image in new RGB space. Visual and subjective investigation is compared with existing methods which demonstrate that our proposed technique gives high structural data content with high spatial and spectral resolution compared withearlier methods.

Sign in / Sign up

Export Citation Format

Share Document