scholarly journals Dissecting Deep Learning Networks—Visualizing Mutual Information

Entropy ◽  
2018 ◽  
Vol 20 (11) ◽  
pp. 823 ◽  
Author(s):  
Hui Fang ◽  
Victoria Wang ◽  
Motonori Yamaguchi
Keyword(s):  
Deep Learning ◽  
Mutual Information ◽  
Network Performance ◽  
Classification Performance ◽  
Knowledge Gap ◽  
Accuracy Evaluation ◽  
Learning Networks ◽  
Theoretical Understanding ◽  
Effective Network ◽  
Visualization Techniques

Deep Learning (DL) networks are recent revolutionary developments in artificial intelligence research. Typical networks are stacked by groups of layers that are further composed of many convolutional kernels or neurons. In network design, many hyper-parameters need to be defined heuristically before training in order to achieve high cross-validation accuracies. However, accuracy evaluation from the output layer alone is not sufficient to specify the roles of the hidden units in associated networks. This results in a significant knowledge gap between DL’s wider applications and its limited theoretical understanding. To narrow the knowledge gap, our study explores visualization techniques to illustrate the mutual information (MI) in DL networks. The MI is a theoretical measurement, reflecting the relationship between two sets of random variables even if their relationship is highly non-linear and hidden in high-dimensional data. Our study aims to understand the roles of DL units in classification performance of the networks. Via a series of experiments using several popular DL networks, it shows that the visualization of MI and its change patterns between the input/output with the hidden layers and basic units can facilitate a better understanding of these DL units’ roles. Our investigation on network convergence suggests a more objective manner to potentially evaluate DL networks. Furthermore, the visualization provides a useful tool to gain insights into the network performance, and thus to potentially facilitate the design of better network architectures by identifying redundancy and less-effective network units.

scholarly journals Architectural Distortion-Based Digital Mammograms Classification Using Depth Wise Convolutional Neural Network

Biology ◽  
2021 ◽  
Vol 11 (1) ◽  
pp. 15
Author(s):  
Khalil ur Rehman ◽  
Jianqiang Li ◽  
Yan Pei ◽  
Anaa Yasin ◽  
Saqib Ali ◽  
...  
Keyword(s):  
Computer Vision ◽  
Deep Learning ◽  
Diagnostic System ◽  
Region Of Interest ◽  
Noise Removal ◽  
Classification Performance ◽  
Architectural Distortion ◽  
Computer Algorithms ◽  
Learning Networks ◽  
Classification Framework

Architectural distortion is the third most suspicious appearance on a mammogram representing abnormal regions. Architectural distortion (AD) detection from mammograms is challenging due to its subtle and varying asymmetry on breast mass and small size. Automatic detection of abnormal ADs regions in mammograms using computer algorithms at initial stages could help radiologists and doctors. The architectural distortion star shapes ROIs detection, noise removal, and object location, affecting the classification performance, reducing accuracy. The computer vision-based technique automatically removes the noise and detects the location of objects from varying patterns. The current study investigated the gap to detect architectural distortion ROIs (region of interest) from mammograms using computer vision techniques. Proposed an automated computer-aided diagnostic system based on architectural distortion using computer vision and deep learning to predict breast cancer from digital mammograms. The proposed mammogram classification framework pertains to four steps such as image preprocessing, augmentation and image pixel-wise segmentation. Architectural distortion ROI`s detection, training deep learning, and machine learning networks to classify AD`s ROIs into malignant and benign classes. The proposed method has been evaluated on three databases, the PINUM, the CBIS-DDSM, and the DDSM mammogram images, using computer vision and depth-wise 2D V-net 64 convolutional neural networks and achieved 0.95, 0.97, and 0.98 accuracies, respectively. Experimental results reveal that our proposed method outperforms as compared with the ShuffelNet, MobileNet, SVM, K-NN, RF, and previous studies.

scholarly journals Automated Classification and Segmentation in Colorectal Images Based on Self-Paced Transfer Network

2021 ◽  
Vol 2021 ◽  
pp. 1-7
Author(s):  
Yao Yao ◽  
Shuiping Gou ◽  
Ru Tian ◽  
Xiangrong Zhang ◽  
Shuixiang He
Keyword(s):  
Deep Learning ◽  
Classification Accuracy ◽  
Network Performance ◽  
Early Stage ◽  
Classification Performance ◽  
Disease Classification ◽  
Colorectal Disease ◽  
Automated Classification ◽  
Initial Network ◽  
Colorectal Diseases

Colorectal imaging improves on diagnosis of colorectal diseases by providing colorectal images. Manual diagnosis of colorectal disease is labor-intensive and time-consuming. In this paper, we present a method for automatic colorectal disease classification and segmentation. Because of label unbalanced and difficult colorectal data, the classification based on self-paced transfer VGG network (STVGG) is proposed. ImageNet pretraining network parameters are transferred to VGG network with training colorectal data to acquire good initial network performance. And self-paced learning is used to optimize the network so that the classification performance of label unbalanced and difficult samples is improved. In order to assist the colonoscopist to accurately determine whether the polyp needs surgical resection, feature of trained STVGG model is shared to Unet segmentation network as the encoder part and to avoid repeat learning of polyp segmentation model. The experimental results on 3061 colorectal images illustrated that the proposed method obtained higher classification accuracy (96%) and segmentation performance compared with a few other methods. The polyp can be segmented accurately from around tissues by the proposed method. The segmentation results underpin the potential of deep learning methods for assisting colonoscopist in identifying polyps and enabling timely resection of these polyps at an early stage.

scholarly journals RESEARCH ON SE-INCEPTION IN HIGH-RESOLUTION REMOTE SENSING IMAGE CLASSIFICATION

2020 ◽  
Vol XLII-3/W10 ◽  
pp. 539-545
Author(s):  
Z. L. Cai ◽  
Q. Weng ◽  
S. Z. Ye
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
High Resolution ◽  
Transfer Learning ◽  
Network Performance ◽  
Learning Strategy ◽  
Remote Sensing Image ◽  
Classification Model ◽  
Learning Networks ◽  
Scene Classification

Abstract. With the deepening research and cross-fusion in the modern remote sensing image area, the classification of high spatial resolution remote sensing images has captured the attention of the researchers in the field of remote sensing. However, due to the serious phenomenon of “same object, different spectrum” and “same spectrum, different object” of high-resolution remote sensing image, the traditional classification strategy is hard to handle this challenge. In this paper, a remote sensing image scene classification model based on SENet and Inception-V3 is proposed by utilizing the deep learning method and transfer learning strategy. The model first adds a dropout layer before the full connection layer of the original Inception-V3 model to avoid over-fitting. Then we embed the SENet module into the Inception-V3 model for optimizing the network performance. In this paper, global average pooling is used as squeeze operation, and then two fully connected layers are used to construct a bottleneck structure. The model proposed in this paper is more non-linear, can better fit the complex correlation between channels, and greatly reduces the amount of parameters and computation. In the training process, this paper adopts the transfer learning strategy, makes full use of existing models and knowledge, improves training efficiency, and finally obtains scene classification results. The experimental results based on AID high-score remote sensing scene images show that SE-Inception has faster convergence speed and more stable training effect than the original Inception-V3 training. Compared with other traditional methods and deep learning networks, the improved model proposed in this paper has greater accuracy improvement.

Arrhythmia Detection Using Gated Recurrent Unit Network with ECG Signals

2020 ◽  
Vol 10 (3) ◽  
pp. 750-757
Author(s):  
Gang Xu ◽  
Guangxin Xing ◽  
Juanjuan Jiang ◽  
Jian Jiang ◽  
Yongsheng Ke
Keyword(s):  
Deep Learning ◽  
Visual Analysis ◽  
Classification Performance ◽  
Ecg Signal ◽  
Learning Networks ◽  
Arrhythmia Detection ◽  
Ecg Signals ◽  
Detection Model ◽  
High Degree ◽  
Gated Recurrent Unit

Background: Arrhythmia is a kind of heart disorder characterized by irregular heartbeats which can be detected with Electrocardiographic (ECG) signals. Accurate and early detection along with differentiation of arrhythmias is of great importance in a clinical setting. However, visual analysis of ECG signal is a challenging and timeconsuming work. We have developed an automatic arrhythmia detection model with deep learning framework to expedite the diagnosis of arrhythmia with a high degree of accuracy. Methods: We proposed a novel automatic arrhythmia detection model utilizing a combination of 1D convolutional neural network (1D-CNN) and Gated Recurrent Unit (GRU) network for the diagnosis of five different arrhythmia on ECG signals taken from the MITBIT arrhythmia physio bank database. Results: The proposed system showed a high classification performance in handling variable length ECG signal data, achieving an accuracy rate of 99.45%, sensitivity of 98.35% and specificity of 99.21% and a F1-Score of 98.95% using a five-fold cross validation strategy. Conclusions: Combining 1D-CNN and GRU networks yielded a higher degree of accuracy compared with other deep learning networks. Our proposed arrhythmia detection method may be a powerful tool to aid clinicians in accurately detecting common arrhythmias on routine ECG screening.

scholarly journals Binary File’s Visualization and Entropy Features Analysis Combined with Multiple Deep Learning Networks for Malware Classification

2020 ◽  
Vol 2020 ◽  
pp. 1-19
Author(s):  
Hui Guo ◽  
Shuguang Huang ◽  
Cheng Huang ◽  
Fan Shi ◽  
Min Zhang ◽  
...  
Keyword(s):  
Deep Learning ◽  
Visual Features ◽  
Variant Classification ◽  
Classification Methods ◽  
Learning Networks ◽  
Visual Classification ◽  
Malware Classification ◽  
Series Of Experiments ◽  
Visualization Techniques

In recent years, the research on malware variant classification has attracted much more attention. However, there are still many challenges, including the low accuracy of classification of samples of similar malware families, high time, and resource consumption. This paper proposes a new method of malware classification based on multiple visual features of malware and deep learning algorithms. In prior research, visualization techniques and entropy demonstrated exemplary performance in many areas. This paper extracts numerous visual features from the raw bytes and entropy sequence of the malware, which makes it more sensitive to malware samples of similar families and endows it the ability to classify malware variants more accurately. To evaluate the proposed method, this paper conducted a series of experiments on two malware datasets with a total of more than 20,000 samples provided by the Malware Research Lab and Microsoft Research. Through experiments, the method showed its superiority compared with some leading malware visual classification methods, achieving good performance on the accuracy with at least 1% improvement. The accuracy of the method even could reach 99.73% and 99.54%, respectively, on the two datasets.

A Study on the Auxiliary Diagnosis of Thyroid Disease Images Based on Multiple Dimensional Deep Learning Algorithms

2020 ◽  
Vol 16 (3) ◽  
pp. 199-205
Author(s):  
Yuejun Liu ◽  
Yifei Xu ◽  
Xiangzheng Meng ◽  
Xuguang Wang ◽  
Tianxu Bai
Keyword(s):  
Deep Learning ◽  
Learning Algorithms ◽  
Region Of Interest ◽  
Classification Performance ◽  
Thyroid Diseases ◽  
Great Success ◽  
Learning Models ◽  
Good Classification Performance ◽  
Spect Images

Background: Medical imaging plays an important role in the diagnosis of thyroid diseases. In the field of machine learning, multiple dimensional deep learning algorithms are widely used in image classification and recognition, and have achieved great success. Objective: The method based on multiple dimensional deep learning is employed for the auxiliary diagnosis of thyroid diseases based on SPECT images. The performances of different deep learning models are evaluated and compared. Methods: Thyroid SPECT images are collected with three types, they are hyperthyroidism, normal and hypothyroidism. In the pre-processing, the region of interest of thyroid is segmented and the amount of data sample is expanded. Four CNN models, including CNN, Inception, VGG16 and RNN, are used to evaluate deep learning methods. Results: Deep learning based methods have good classification performance, the accuracy is 92.9%-96.2%, AUC is 97.8%-99.6%. VGG16 model has the best performance, the accuracy is 96.2% and AUC is 99.6%. Especially, the VGG16 model with a changing learning rate works best. Conclusion: The standard CNN, Inception, VGG16, and RNN four deep learning models are efficient for the classification of thyroid diseases with SPECT images. The accuracy of the assisted diagnostic method based on deep learning is higher than that of other methods reported in the literature.

scholarly journals UAVs in rail damage image diagnostics supported by deep-learning networks

2021 ◽  
Vol 11 (1) ◽  
pp. 339-348
Author(s):  
Piotr Bojarczak ◽  
Piotr Lesiak
Keyword(s):  
Deep Learning ◽  
Head Width ◽  
Learning Networks ◽  
Image Brightness ◽  
Learning Network ◽  
Rail Head ◽  
Image Recording ◽  
Python Language ◽  
Deep Learning Network ◽  
Efficiency Rate

Abstract The article uses images from Unmanned Aerial Vehicles (UAVs) for rail diagnostics. The main advantage of such a solution compared to traditional surveys performed with measuring vehicles is the elimination of decreased train traffic. The authors, in the study, limited themselves to the diagnosis of hazardous split defects in rails. An algorithm has been proposed to detect them with an efficiency rate of about 81% for defects not less than 6.9% of the rail head width. It uses the FCN-8 deep-learning network, implemented in the Tensorflow environment, to extract the rail head by image segmentation. Using this type of network for segmentation increases the resistance of the algorithm to changes in the recorded rail image brightness. This is of fundamental importance in the case of variable conditions for image recording by UAVs. The detection of these defects in the rail head is performed using an algorithm in the Python language and the OpenCV library. To locate the defect, it uses the contour of a separate rail head together with a rectangle circumscribed around it. The use of UAVs together with artificial intelligence to detect split defects is an important element of novelty presented in this work.

Sign in / Sign up

Export Citation Format

Share Document