scholarly journals Multi-Modality Medical Image Fusion Using Convolutional Neural Network and Contrast Pyramid

Sensors ◽  
2020 ◽  
Vol 20 (8) ◽  
pp. 2169 ◽  
Author(s):  
Kunpeng Wang ◽  
Mingyao Zheng ◽  
Hongyan Wei ◽  
Guanqiu Qi ◽  
Yuanyuan Li
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Image Fusion ◽  
Medical Image ◽  
Source Image ◽  
Fusion Algorithm ◽  
Convolutional Network ◽  
Structure Information ◽  
Medical Image Fusion ◽  
Fused Image

Medical image fusion techniques can fuse medical images from different morphologies to make the medical diagnosis more reliable and accurate, which play an increasingly important role in many clinical applications. To obtain a fused image with high visual quality and clear structure details, this paper proposes a convolutional neural network (CNN) based medical image fusion algorithm. The proposed algorithm uses the trained Siamese convolutional network to fuse the pixel activity information of source images to realize the generation of weight map. Meanwhile, a contrast pyramid is implemented to decompose the source image. According to different spatial frequency bands and a weighted fusion operator, source images are integrated. The results of comparative experiments show that the proposed fusion algorithm can effectively preserve the detailed structure information of source images and achieve good human visual effects.

scholarly journals Multimodal Medical Image Fusion Based on Multiple Latent Low-Rank Representation

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Xi-Cheng Lou ◽  
Xin Feng
Keyword(s):  
Image Fusion ◽  
Medical Image ◽  
Hadamard Product ◽  
Subjective Evaluation ◽  
Low Rank ◽  
Source Image ◽  
Fusion Algorithm ◽  
Medical Image Fusion ◽  
Fused Image ◽  
Low Rank Representation

A multimodal medical image fusion algorithm based on multiple latent low-rank representation is proposed to improve imaging quality by solving fuzzy details and enhancing the display of lesions. Firstly, the proposed method decomposes the source image repeatedly using latent low-rank representation to obtain several saliency parts and one low-rank part. Secondly, the VGG-19 network identifies the low-rank part’s features and generates the weight maps. Then, the fused low-rank part can be obtained by making the Hadamard product of the weight maps and the source images. Thirdly, the fused saliency parts can be obtained by selecting the max value. Finally, the fused saliency parts and low-rank part are superimposed to obtain the fused image. Experimental results show that the proposed method is superior to the traditional multimodal medical image fusion algorithms in the subjective evaluation and objective indexes.

Medical Image Fusion in Wavelet and Ridgelet Domains

2017 ◽  
pp. 711-723
Author(s):  
Vikrant Bhateja ◽  
Abhinav Krishn ◽  
Himanshi Patel ◽  
Akanksha Sahu
Keyword(s):  
Image Fusion ◽  
Information Content ◽  
Medical Image ◽  
Similarity Index ◽  
Principal Component ◽  
Structural Similarity ◽  
Fusion Algorithm ◽  
Medical Image Fusion ◽  
Multi Scale ◽  
Fused Image

Medical image fusion facilitates the retrieval of complementary information from medical images and has been employed diversely for computer-aided diagnosis of life threatening diseases. Fusion has been performed using various approaches such as Pyramidal, Multi-resolution, multi-scale etc. Each and every approach of fusion depicts only a particular feature (i.e. the information content or the structural properties of an image). Therefore, this paper presents a comparative analysis and evaluation of multi-modal medical image fusion methodologies employing wavelet as a multi-resolution approach and ridgelet as a multi-scale approach. The current work tends to highlight upon the utility of these approaches according to the requirement of features in the fused image. Principal Component Analysis (PCA) based fusion algorithm has been employed in both ridgelet and wavelet domains for purpose of minimisation of redundancies. Simulations have been performed for different sets of MR and CT-scan images taken from ‘The Whole Brain Atlas'. The performance evaluation has been carried out using different parameters of image quality evaluation like: Entropy (E), Fusion Factor (FF), Structural Similarity Index (SSIM) and Edge Strength (QFAB). The outcome of this analysis highlights the trade-off between the retrieval of information content and the morphological details in finally fused image in wavelet and ridgelet domains.

scholarly journals Local energy match based non-sub sampled contourlet Transform for multi modal medical image fusion

2018 ◽  
Vol 7 (2.31) ◽  
pp. 165
Author(s):  
M Shyamala Devi ◽  
P Balamurugan
Keyword(s):  
Image Fusion ◽  
High Frequency ◽  
Medical Image ◽  
Gabor Filter ◽  
Low Frequency ◽  
Local Energy ◽  
Fusion Algorithm ◽  
Medical Image Fusion ◽  
Fused Image ◽  
Image Averaging

Image processing technology requires moreover the full image or the part of image which is to be processed from the user’s point of view like the radius of object etc. The main purpose of fusion is to diminish dissimilar error between the fused image and the input images. With respect to the medical diagnosis, the edges and outlines of the concerned objects is more important than extra information. So preserving the edge features of the image is worth for investigating the image fusion. The image with higher contrast contains more edge-like features. Here we propose a new medical image fusion scheme namely Local Energy Match NSCT based on discrete contourlet transformation, which is constructive to give the details of curve edges. It is used to progress the edge information of fused image by dropping the distortion. This transformation lead to crumbling of multimodal image addicted to finer and coarser details and finest details will be decayed into unusual resolution in dissimilar orientation. The input multimodal images namely CT and MRI images are first transformed by Non Sub sampled Contourlet Transformation (NSCT) which decomposes the image into low frequency and high frequency elements. In our system, the Low frequency coefficient of the image is fused by image averaging and Gabor filter bank algorithm. The processed High frequency coefficients of the image are fused by image averaging and gradient based fusion algorithm. Then the fused image is obtained by inverse NSCT with local energy match based coefficients. To evaluate the image fusion accuracy, Peak Signal to Noise Ratio (PSNR), Root Mean Square Error (RMSE) and Correlation Coefficient parameters are used in this work .

scholarly journals Brain Medical Image Fusion Based on Dual-Branch CNNs in NSST Domain

2020 ◽  
Vol 2020 ◽  
pp. 1-15
Author(s):  
Zhaisheng Ding ◽  
Dongming Zhou ◽  
Rencan Nie ◽  
Ruichao Hou ◽  
Yanyu Liu
Keyword(s):  
Image Fusion ◽  
High Frequency ◽  
Medical Image ◽  
Low Frequency ◽  
Structural Features ◽  
Functional Information ◽  
Convolutional Network ◽  
Structure Information ◽  
Medical Image Fusion ◽  
Research Challenge

Computed tomography (CT) images show structural features, while magnetic resonance imaging (MRI) images represent brain tissue anatomy but do not contain any functional information. How to effectively combine the images of the two modes has become a research challenge. In this paper, a new framework for medical image fusion is proposed which combines convolutional neural networks (CNNs) and non-subsampled shearlet transform (NSST) to simultaneously cover the advantages of them both. This method effectively retains the functional information of the CT image and reduces the loss of brain structure information and spatial distortion of the MRI image. In our fusion framework, the initial weights integrate the pixel activity information from two source images that is generated by a dual-branch convolutional network and is decomposed by NSST. Firstly, the NSST is performed on the source images and the initial weights to obtain their low-frequency and high-frequency coefficients. Then, the first component of the low-frequency coefficients is fused by a novel fusion strategy, which simultaneously copes with two key issues in the fusion processing which are named energy conservation and detail extraction. The second component of the low-frequency coefficients is fused by the strategy that is designed according to the spatial frequency of the weight map. Moreover, the high-frequency coefficients are fused by the high-frequency components of the initial weight. Finally, the final image is reconstructed by the inverse NSST. The effectiveness of the proposed method is verified using pairs of multimodality images, and the sufficient experiments indicate that our method performs well especially for medical image fusion.

scholarly journals Multi-Modal Medical Image Fusion Based on FusionNet in YIQ Color Space

Entropy ◽  
2020 ◽  
Vol 22 (12) ◽  
pp. 1423
Author(s):  
Kai Guo ◽  
Xiongfei Li ◽  
Hongrui Zang ◽  
Tiehu Fan
Keyword(s):  
Image Fusion ◽  
Medical Image ◽  
Medical Information ◽  
Color Space ◽  
Structural Similarity ◽  
Source Image ◽  
Medical Image Fusion ◽  
Structure Recognition ◽  
Fused Image ◽  
Fusion Framework

In order to obtain the physiological information and key features of source images to the maximum extent, improve the visual effect and clarity of the fused image, and reduce the computation, a multi-modal medical image fusion framework based on feature reuse is proposed. The framework consists of intuitive fuzzy processing (IFP), capture image details network (CIDN), fusion, and decoding. First, the membership function of the image is redefined to remove redundant features and obtain the image with complete features. Then, inspired by DenseNet, we proposed a new encoder to capture all the medical information features in the source image. In the fusion layer, we calculate the weight of each feature graph in the required fusion coefficient according to the trajectory of the feature graph. Finally, the filtered medical information is spliced and decoded to reproduce the required fusion image. In the encoding and image reconstruction networks, the mixed loss function of cross entropy and structural similarity is adopted to greatly reduce the information loss in image fusion. To assess performance, we conducted three sets of experiments on medical images of different grayscales and colors. Experimental results show that the proposed algorithm has advantages not only in detail and structure recognition but also in visual features and time complexity compared with other algorithms.

scholarly journals Medical Image Fusion Based on Sparse Representation and PCNN in NSCT Domain

2018 ◽  
Vol 2018 ◽  
pp. 1-12 ◽  
Author(s):  
Jingming Xia ◽  
Yiming Chen ◽  
Aiyue Chen ◽  
Yicai Chen
Keyword(s):  
Neural Network ◽  
Image Fusion ◽  
Sparse Representation ◽  
High Frequency ◽  
Medical Image ◽  
Information Transfer ◽  
Color Image ◽  
Low Frequency ◽  
Source Image ◽  
Medical Image Fusion

The clinical assistant diagnosis has a high requirement for the visual effect of medical images. However, the low frequency subband coefficients obtained by the NSCT decomposition are not sparse, which is not conducive to maintaining the details of the source image. To solve these problems, a medical image fusion algorithm combined with sparse representation and pulse coupling neural network is proposed. First, the source image is decomposed into low and high frequency subband coefficients by NSCT transform. Secondly, the K singular value decomposition (K-SVD) method is used to train the low frequency subband coefficients to get the overcomplete dictionary D, and the orthogonal matching pursuit (OMP) algorithm is used to sparse the low frequency subband coefficients to complete the fusion of the low frequency subband sparse coefficients. Then, the pulse coupling neural network (PCNN) is excited by the spatial frequency of the high frequency subband coefficients, and the fusion coefficients of the high frequency subband coefficients are selected according to the number of ignition times. Finally, the fusion medical image is reconstructed by NSCT inverter. The experimental results and analysis show that the algorithm of gray and color image fusion is about 34% and 10% higher than the contrast algorithm in the edge information transfer factor QAB/F index, and the performance of the fusion result is better than the existing algorithm.

Hybrid Multimodality Medical Image Fusion Using Various Fusion Techniques With Quantitative and Qualitative Analysis

2019 ◽  
pp. 206-233 ◽  
Author(s):  
Rajalingam B. ◽  
Priya R. ◽  
Bhavani R.
Keyword(s):  
Computed Tomography ◽  
Image Fusion ◽  
Medical Image ◽  
Single Photon ◽  
Photon Emission ◽  
Fusion Algorithm ◽  
Medical Image Fusion ◽  
Fused Image ◽  
Visual Description ◽  
The Individual

In this chapter, different types of image fusion techniques have been studied and evaluated in the medical applications. The ultimate goal of this proposed method is to obtain the fused image without any loss of similar information and preserve all special features present in the input medical images. This method is used to improve the fused image quality for better diagnosis of critical disease analysis. The fused hybrid multimodal medical image should convey better visual description than the individual input images. This chapter proposes the method for multimodal medical image fusion using the hybrid fusion algorithm. The computed tomography, magnetic resonance imaging, positron emission tomography, and single photon emission computed tomography are the input images used for this experimental work. In this chapter, experimental results discovered that the proposed techniques provide better visualization of fused image and gives the superior results compared to various existing traditional algorithms.

Sign in / Sign up

Export Citation Format

Share Document