scholarly journals Deep Learning-Based Stacked Denoising and Autoencoder for ECG Heartbeat Classification

Electronics ◽  
2020 ◽  
Vol 9 (1) ◽  
pp. 135 ◽  
Author(s):  
Siti Nurmaini ◽  
Annisa Darmawahyuni ◽  
Akhmad Noviar Sakti Mukti ◽  
Muhammad Naufal Rachmatullah ◽  
Firdaus Firdaus ◽  
...  
Keyword(s):  
Deep Learning ◽  
Signal To Noise Ratio ◽  
Stress Test ◽  
Feature Learning ◽  
Feature Representation ◽  
Training Data ◽  
Fine Tuning ◽  
Noninvasive Test ◽  
Heartbeat Classification ◽  
Unseen Data

The electrocardiogram (ECG) is a widely used, noninvasive test for analyzing arrhythmia. However, the ECG signal is prone to contamination by different kinds of noise. Such noise may cause deformation on the ECG heartbeat waveform, leading to cardiologists’ mislabeling or misinterpreting heartbeats due to varying types of artifacts and interference. To address this problem, some previous studies propose a computerized technique based on machine learning (ML) to distinguish between normal and abnormal heartbeats. Unfortunately, ML works on a handcrafted, feature-based approach and lacks feature representation. To overcome such drawbacks, deep learning (DL) is proposed in the pre-training and fine-tuning phases to produce an automated feature representation for multi-class classification of arrhythmia conditions. In the pre-training phase, stacked denoising autoencoders (DAEs) and autoencoders (AEs) are used for feature learning; in the fine-tuning phase, deep neural networks (DNNs) are implemented as a classifier. To the best of our knowledge, this research is the first to implement stacked autoencoders by using DAEs and AEs for feature learning in DL. Physionet’s well-known MIT-BIH Arrhythmia Database, as well as the MIT-BIH Noise Stress Test Database (NSTDB). Only four records are used from the NSTDB dataset: 118 24 dB, 118 −6 dB, 119 24 dB, and 119 −6 dB, with two levels of signal-to-noise ratio (SNRs) at 24 dB and −6 dB. In the validation process, six models are compared to select the best DL model. For all fine-tuned hyperparameters, the best model of ECG heartbeat classification achieves an accuracy, sensitivity, specificity, precision, and F1-score of 99.34%, 93.83%, 99.57%, 89.81%, and 91.44%, respectively. As the results demonstrate, the proposed DL model can extract high-level features not only from the training data but also from unseen data. Such a model has good application prospects in clinical practice.

scholarly journals Deep Learning-Based Differentiation between Mucinous Cystic Neoplasm and Serous Cystic Neoplasm in the Pancreas Using Endoscopic Ultrasonography

Diagnostics ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. 1052
Author(s):  
Leang Sim Nguon ◽  
Kangwon Seo ◽  
Jung-Hyun Lim ◽  
Tae-Jun Song ◽  
Sung-Hyun Cho ◽  
...  
Keyword(s):  
Decision Making ◽  
Deep Learning ◽  
Network Model ◽  
Endoscopic Ultrasonography ◽  
Data Augmentation ◽  
Clinical Information ◽  
Training Data ◽  
Fine Tuning ◽  
Cystic Neoplasm ◽  
Cystic Neoplasms

Mucinous cystic neoplasms (MCN) and serous cystic neoplasms (SCN) account for a large portion of solitary pancreatic cystic neoplasms (PCN). In this study we implemented a convolutional neural network (CNN) model using ResNet50 to differentiate between MCN and SCN. The training data were collected retrospectively from 59 MCN and 49 SCN patients from two different hospitals. Data augmentation was used to enhance the size and quality of training datasets. Fine-tuning training approaches were utilized by adopting the pre-trained model from transfer learning while training selected layers. Testing of the network was conducted by varying the endoscopic ultrasonography (EUS) image sizes and positions to evaluate the network performance for differentiation. The proposed network model achieved up to 82.75% accuracy and a 0.88 (95% CI: 0.817–0.930) area under curve (AUC) score. The performance of the implemented deep learning networks in decision-making using only EUS images is comparable to that of traditional manual decision-making using EUS images along with supporting clinical information. Gradient-weighted class activation mapping (Grad-CAM) confirmed that the network model learned the features from the cyst region accurately. This study proves the feasibility of diagnosing MCN and SCN using a deep learning network model. Further improvement using more datasets is needed.

scholarly journals On the objectivity, reliability, and validity of deep learning enabled bioimage analyses

eLife ◽  
2020 ◽  
Vol 9 ◽  
Author(s):  
Dennis Segebarth ◽  
Matthias Griebel ◽  
Nikolai Stein ◽  
Cora R von Collenberg ◽  
Corinna Martin ◽  
...  
Keyword(s):  
Deep Learning ◽  
Signal To Noise Ratio ◽  
Biological Effects ◽  
Reliability And Validity ◽  
Ground Truth ◽  
Training Data ◽  
Model Organisms ◽  
Data Annotation ◽  
Bioimage Analysis ◽  
Model Training

Bioimage analysis of fluorescent labels is widely used in the life sciences. Recent advances in deep learning (DL) allow automating time-consuming manual image analysis processes based on annotated training data. However, manual annotation of fluorescent features with a low signal-to-noise ratio is somewhat subjective. Training DL models on subjective annotations may be instable or yield biased models. In turn, these models may be unable to reliably detect biological effects. An analysis pipeline integrating data annotation, ground truth estimation, and model training can mitigate this risk. To evaluate this integrated process, we compared different DL-based analysis approaches. With data from two model organisms (mice, zebrafish) and five laboratories, we show that ground truth estimation from multiple human annotators helps to establish objectivity in fluorescent feature annotations. Furthermore, ensembles of multiple models trained on the estimated ground truth establish reliability and validity. Our research provides guidelines for reproducible DL-based bioimage analyses.

scholarly journals Empirical Evaluation of the Effect of Optimization and Regularization Techniques on the Generalization Performance of Deep Convolutional Neural Network

2020 ◽  
Vol 10 (21) ◽  
pp. 7817
Author(s):  
Ivana Marin ◽  
Ana Kuzmanic Skelin ◽  
Tamara Grujic
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
Empirical Evaluation ◽  
Training Data ◽  
Generalization Performance ◽  
Deep Model ◽  
Unseen Data ◽  
Regularization Techniques ◽  
Learning Architectures

The main goal of any classification or regression task is to obtain a model that will generalize well on new, previously unseen data. Due to the recent rise of deep learning and many state-of-the-art results obtained with deep models, deep learning architectures have become one of the most used model architectures nowadays. To generalize well, a deep model needs to learn the training data well without overfitting. The latter implies a correlation of deep model optimization and regularization with generalization performance. In this work, we explore the effect of the used optimization algorithm and regularization techniques on the final generalization performance of the model with convolutional neural network (CNN) architecture widely used in the field of computer vision. We give a detailed overview of optimization and regularization techniques with a comparative analysis of their performance with three CNNs on the CIFAR-10 and Fashion-MNIST image datasets.

scholarly journals A general approach for improving deep learning-based medical relation extraction using a pre-trained model and fine-tuning

Database ◽  
2019 ◽  
Vol 2019 ◽  
Author(s):  
Tao Chen ◽  
Mingfen Wu ◽  
Hexi Li
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Relation Extraction ◽  
Training Model ◽  
Biomedical Literature ◽  
Training Data ◽  
Fine Tuning ◽  
Learning Approaches ◽  
Additional Time ◽  
Clinical Records

Abstract The automatic extraction of meaningful relations from biomedical literature or clinical records is crucial in various biomedical applications. Most of the current deep learning approaches for medical relation extraction require large-scale training data to prevent overfitting of the training model. We propose using a pre-trained model and a fine-tuning technique to improve these approaches without additional time-consuming human labeling. Firstly, we show the architecture of Bidirectional Encoder Representations from Transformers (BERT), an approach for pre-training a model on large-scale unstructured text. We then combine BERT with a one-dimensional convolutional neural network (1d-CNN) to fine-tune the pre-trained model for relation extraction. Extensive experiments on three datasets, namely the BioCreative V chemical disease relation corpus, traditional Chinese medicine literature corpus and i2b2 2012 temporal relation challenge corpus, show that the proposed approach achieves state-of-the-art results (giving a relative improvement of 22.2, 7.77, and 38.5% in F1 score, respectively, compared with a traditional 1d-CNN classifier). The source code is available at https://github.com/chentao1999/MedicalRelationExtraction.

Fast Image Search with Pixel-Based Deep Learning Framework via Efficient Compact Binary Code and Addictive Latent Layer

2017 ◽  
Vol 32 (03) ◽  
pp. 1859004 ◽  
Author(s):  
Jun Yi Li ◽  
Jian Hua Li
Keyword(s):  
Deep Learning ◽  
Nearest Neighbor ◽  
Feature Learning ◽  
Nearest Neighbor Search ◽  
Feature Representation ◽  
Image Feature ◽  
Retrieval Algorithm ◽  
Image Search ◽  
Learning Framework ◽  
Compact Binary

As we know, the nearest neighbor search is a good and effective method for good-sized image search. This paper mainly introduced how to learn an outstanding image feature representation form and a series of compact binary Hash coding functions under deep learning framework. Our concept is that binary codes can be obtained using a hidden layer to present some latent concepts dominating the class labels with usable data labels. Our method is effective in obtaining hash codes and image representations, so it is suitable for good-sized dataset. It is demonstrated in our experiment that the performances of the proposed algorithms were then verified on three different databases, MNIST, CIFAR-10 and Caltech-101. The experimental results reveal that two-proposed image Hash retrieval algorithm based on pixel-level automatic feature learning show higher search accuracy than the other algorithms; moreover, these two algorithms were proved to be more favorable in scalability and generality.

scholarly journals DEEP LEARNING BASED ROOF TYPE CLASSIFICATION USING VERY HIGH RESOLUTION AERIAL IMAGERY

2021 ◽  
Vol XLIII-B3-2021 ◽  
pp. 55-60
Author(s):  
M. Buyukdemircioglu ◽  
R. Can ◽  
S. Kocaman
Keyword(s):  
Deep Learning ◽  
High Resolution ◽  
Urban Areas ◽  
Image Features ◽  
Training Data ◽  
Fine Tuning ◽  
Computer Hardware ◽  
Geographical Information ◽  
Training Dataset ◽  
Very High

Abstract. Automatic detection, segmentation and reconstruction of buildings in urban areas from Earth Observation (EO) data are still challenging for many researchers. Roof is one of the most important element in a building model. The three-dimensional geographical information system (3D GIS) applications generally require the roof type and roof geometry for performing various analyses on the models, such as energy efficiency. The conventional segmentation and classification methods are often based on features like corners, edges and line segments. In parallel to the developments in computer hardware and artificial intelligence (AI) methods including deep learning (DL), image features can be extracted automatically. As a DL technique, convolutional neural networks (CNNs) can also be used for image classification tasks, but require large amount of high quality training data for obtaining accurate results. The main aim of this study was to generate a roof type dataset from very high-resolution (10 cm) orthophotos of Cesme, Turkey, and to classify the roof types using a shallow CNN architecture. The training dataset consists 10,000 roof images and their labels. Six roof type classes such as flat, hip, half-hip, gable, pyramid and complex roofs were used for the classification in the study area. The prediction performance of the shallow CNN model used here was compared with the results obtained from the fine-tuning of three well-known pre-trained networks, i.e. VGG-16, EfficientNetB4, ResNet-50. The results show that although our CNN has slightly lower performance expressed with the overall accuracy, it is still acceptable for many applications using sparse data.

scholarly journals Design of an Novel Spectrum Sensing Scheme Based on Long Short-Term Memory and Experimental Validation

2021 ◽  
Vol 15 ◽  
pp. 26-32
Author(s):  
Nupur Choudhury ◽  
Kandarpa Kumar Sarma ◽  
Chinmoy Kalita ◽  
Aradhana Misra
Keyword(s):  
Deep Learning ◽  
Spectrum Sensing ◽  
Short Term Memory ◽  
Signal To Noise Ratio ◽  
Feature Learning ◽  
Noise Model ◽  
Critical Element ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Spectrum sensing allows cognitive radio systems to detect relevant signals in despite the presence of severe interference. Most of the existing spectrum sensing techniques use a particular signal-noise model with certain assumptions and derive certain detection performance. To deal with this uncertainty, learning based approaches are being adopted and more recently deep learning based tools have become popular. Here, we propose an approach of spectrum sensing which is based on long short term memory (LSTM) which is a critical element of deep learning networks (DLN). Use of LSTM facilitates implicit feature learning from spectrum data. The DLN is trained using several features and the performance of the proposed sensing technique is validated with the help of an empirical testbed setup using Adalm Pluto. The testbed is trained to acquire the primary signal of a real world radio broadcast taking place using FM. Experimental data show that even at low signal to noise ratio, our approach performs well in terms of detection and classification accuracies, as compared to current spectrum sensing methods.

scholarly journals Identifying indicator species in ecological habitats using Deep Optimal Feature Learning

PLoS ONE ◽  
2021 ◽  
Vol 16 (9) ◽  
pp. e0256782
Author(s):  
Yiting Tsai ◽  
Susan A. Baldwin ◽  
Bhushan Gopaluni
Keyword(s):  
Deep Learning ◽  
Indicator Species ◽  
Prediction Accuracy ◽  
Species Abundance ◽  
Keystone Species ◽  
Feature Learning ◽  
Training Data ◽  
Support Vector ◽  
Vector Machines ◽  
Domain Expertise

Much of the current research on supervised modelling is focused on maximizing outcome prediction accuracy. However, in engineering disciplines, an arguably more important goal is that of feature extraction, the identification of relevant features associated with the various outcomes. For instance, in microbial communities, the identification of keystone species can often lead to improved prediction of future behavioral shifts. This paper proposes a novel feature extractor based on Deep Learning, which is largely agnostic to underlying assumptions regarding the training data. Starting from a collection of microbial species abundance counts, the Deep Learning model first trains itself to classify the selected distinct habitats. It then identifies indicator species associated with the habitats. The results are then compared and contrasted with those obtained by traditional statistical techniques. The indicator species are similar when compared at top taxonomic levels such as Domain and Phylum, despite visible differences in lower levels such as Class and Order. More importantly, when our estimated indicators are used to predict final habitat labels using simpler models (such as Support Vector Machines and traditional Artificial Neural Networks), the prediction accuracy is improved. Overall, this study serves as a preliminary step that bridges modern, black-box Machine Learning models with traditional, domain expertise-rich techniques.

scholarly journals Deep learning-based enhancement of epigenomics data with AtacWorks

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Avantika Lal ◽  
Zachary D. Chiang ◽  
Nikolai Yakovenko ◽  
Fabiana M. Duarte ◽  
Johnny Israeli ◽  
...  
Keyword(s):  
Deep Learning ◽  
Signal To Noise Ratio ◽  
Cell Types ◽  
Chromatin Accessibility ◽  
Training Data ◽  
Regulatory Regions ◽  
Hematopoietic Stem ◽  
Sequencing Coverage ◽  
Lineage Priming ◽  
Low Coverage

AbstractATAC-seq is a widely-applied assay used to measure genome-wide chromatin accessibility; however, its ability to detect active regulatory regions can depend on the depth of sequencing coverage and the signal-to-noise ratio. Here we introduce AtacWorks, a deep learning toolkit to denoise sequencing coverage and identify regulatory peaks at base-pair resolution from low cell count, low-coverage, or low-quality ATAC-seq data. Models trained by AtacWorks can detect peaks from cell types not seen in the training data, and are generalizable across diverse sample preparations and experimental platforms. We demonstrate that AtacWorks enhances the sensitivity of single-cell experiments by producing results on par with those of conventional methods using ~10 times as many cells, and further show that this framework can be adapted to enable cross-modality inference of protein-DNA interactions. Finally, we establish that AtacWorks can enable new biological discoveries by identifying active regulatory regions associated with lineage priming in rare subpopulations of hematopoietic stem cells.

scholarly journals An Improved Deep Learning Model for Traffic Crash Prediction

2018 ◽  
Vol 2018 ◽  
pp. 1-13 ◽  
Author(s):  
Chunjiao Dong ◽  
Chunfu Shao ◽  
Juan Li ◽  
Zhihua Xiong
Keyword(s):  
Deep Learning ◽  
Feature Learning ◽  
Learning Model ◽  
Fine Tuning ◽  
Crash Prediction ◽  
Traffic Crash ◽  
Explanatory Variables ◽  
Feature Representations ◽  
Proposed Model ◽  
Deep Learning Model

Machine-learning technology powers many aspects of modern society. Compared to the conventional machine learning techniques that were limited in processing natural data in the raw form, deep learning allows computational models to learn representations of data with multiple levels of abstraction. In this study, an improved deep learning model is proposed to explore the complex interactions among roadways, traffic, environmental elements, and traffic crashes. The proposed model includes two modules, an unsupervised feature learning module to identify functional network between the explanatory variables and the feature representations and a supervised fine tuning module to perform traffic crash prediction. To address the unobserved heterogeneity issues in the traffic crash prediction, a multivariate negative binomial (MVNB) model is embedding into the supervised fine tuning module as a regression layer. The proposed model was applied to the dataset that was collected from Knox County in Tennessee to validate the performances. The results indicate that the feature learning module identifies relational information between the explanatory variables and the feature representations, which reduces the dimensionality of the input and preserves the original information. The proposed model that includes the MVNB regression layer in the supervised fine tuning module can better account for differential distribution patterns in traffic crashes across injury severities and provides superior traffic crash predictions. The findings suggest that the proposed model is a superior alternative for traffic crash predictions and the average accuracy of the prediction that was measured by RMSD can be improved by 84.58% and 158.27% compared to the deep learning model without the regression layer and the SVM model, respectively.

Sign in / Sign up

Export Citation Format

Share Document