scholarly journals A Transfer Learning Approach for Lumbar Spine Disc State Classification

Electronics ◽  
2021 ◽  
Vol 11 (1) ◽  
pp. 85
Author(s):  
Ali Al-kubaisi ◽  
Nasser N. Khamiss
Keyword(s):  
Deep Learning ◽  
Lumbar Spine ◽  
Transfer Learning ◽  
Region Of Interest ◽  
Training Data ◽  
Learning Tools ◽  
Analysis Process ◽  
Lower Back ◽  
State Classification ◽  
Classification Tasks

Recently, deep learning algorithms have become one of the most popular methods and forms of algorithms used in the medical imaging analysis process. Deep learning tools provide accuracy and speed in the process of diagnosing and classifying lumbar spine problems. Disk herniation and spinal stenosis are two of the most common lower back diseases. The process of diagnosing pain in the lower back can be considered costly in terms of time and available expertise. In this paper, we used multiple approaches to overcome the problem of lack of training data in disc state classification and to enhance the performance of disc state classification tasks. To achieve this goal, transfer learning from different datasets and a proposed region of interest (ROI) technique were implemented. It has been demonstrated that using transfer learning from the same domain as the target dataset may increase performance dramatically. Applying the ROI method improved the disc state classification results in VGG19 2%, ResNet50 16%, MobileNetV2 5%, and VGG16 2%. The results improved VGG16 4% and in VGG19 6%, compared with the transfer from ImageNet. Moreover, it has been stated that the closer the data to be classified is to the data that the system trained on, the better the achieved results will be.

scholarly journals Copy-Move Forgery Detection (CMFD) Using Deep Learning for Image and Video Forensics

2021 ◽  
Vol 7 (3) ◽  
pp. 59
Author(s):  
Yohanna Rodriguez-Ortega ◽  
Dora M. Ballesteros ◽  
Diego Renza
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
Training Data ◽  
Video Editing ◽  
Forgery Detection ◽  
Copy Move Forgery Detection ◽  
The Impact ◽  
And Training ◽  
Selection Of ◽  
Traditional Image

With the exponential growth of high-quality fake images in social networks and media, it is necessary to develop recognition algorithms for this type of content. One of the most common types of image and video editing consists of duplicating areas of the image, known as the copy-move technique. Traditional image processing approaches manually look for patterns related to the duplicated content, limiting their use in mass data classification. In contrast, approaches based on deep learning have shown better performance and promising results, but they present generalization problems with a high dependence on training data and the need for appropriate selection of hyperparameters. To overcome this, we propose two approaches that use deep learning, a model by a custom architecture and a model by transfer learning. In each case, the impact of the depth of the network is analyzed in terms of precision (P), recall (R) and F1 score. Additionally, the problem of generalization is addressed with images from eight different open access datasets. Finally, the models are compared in terms of evaluation metrics, and training and inference times. The model by transfer learning of VGG-16 achieves metrics about 10% higher than the model by a custom architecture, however, it requires approximately twice as much inference time as the latter.

Uncertainty quantification in fault detection using convolutional neural networks

Geophysics ◽  
2021 ◽  
pp. 1-45
Author(s):  
Runhai Feng ◽  
Dario Grana ◽  
Niels Balling
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Reservoir Characterization ◽  
Epistemic Uncertainty ◽  
Confidence Regions ◽  
Training Data ◽  
Model Parameters ◽  
Sigmoid Function ◽  
Learning Tools

Segmentation of faults based on seismic images is an important step in reservoir characterization. With the recent developments of deep-learning methods and the availability of massive computing power, automatic interpretation of seismic faults has become possible. The likelihood of occurrence for a fault can be quantified using a sigmoid function. Our goal is to quantify the fault model uncertainty that is generally not captured by deep-learning tools. We propose to use the dropout approach, a regularization technique to prevent overfitting and co-adaptation in hidden units, to approximate the Bayesian inference and estimate the principled uncertainty over functions. Particularly, the variance of the learned model has been decomposed into aleatoric and epistemic parts. The proposed method is applied to a real dataset from the Netherlands F3 block with two different dropout ratios in convolutional neural networks. The aleatoric uncertainty is irreducible since it relates to the stochastic dependency within the input observations. As the number of Monte-Carlo realizations increases, the epistemic uncertainty asymptotically converges and the model standard deviation decreases, because the variability of model parameters is better simulated or explained with a larger sample size. This analysis can quantify the confidence to use fault predictions with less uncertainty. Additionally, the analysis suggests where more training data are needed to reduce the uncertainty in low confidence regions.

scholarly journals Same Representation, Different Attentions: Shareable Sentence Representation Learning from Multiple Tasks

2018 ◽  
Author(s):  
Renjie Zheng ◽  
Junkun Chen ◽  
Xipeng Qiu
Keyword(s):  
Deep Learning ◽  
Language Processing ◽  
Text Classification ◽  
Representation Learning ◽  
Training Data ◽  
Specific Information ◽  
Distributed Representation ◽  
Source Codes ◽  
Multiple Tasks ◽  
Classification Tasks

Distributed representation plays an important role in deep learning based natural language processing. However, the representation of a sentence often varies in different tasks, which is usually learned from scratch and suffers from the limited amounts of training data. In this paper, we claim that a good sentence representation should be invariant and can benefit the various subsequent tasks. To achieve this purpose, we propose a new scheme of information sharing for multi-task learning. More specifically, all tasks share the same sentence representation and each task can select the task-specific information from the shared sentence representation with attention mechanisms. The query vector of each task's attention could be either static parameters or generated dynamically. We conduct extensive experiments on 16 different text classification tasks, which demonstrate the benefits of our architecture. Source codes of this paper are available on Github.

scholarly journals Task independent transfer learning in EEG deep-learning classification tasks: Sex classification and anti-depressant response prediction

2021 ◽  
Vol 14 (6) ◽  
pp. 1736
Author(s):  
Hamish Binnie ◽  
Kristi Griffiths ◽  
Evian Gordon ◽  
Mayuresh Korgaonkar ◽  
Martijn Arns ◽  
...  
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
Response Prediction ◽  
Classification Tasks

scholarly journals SAR Target Classification Based on Sample Spectral Regularization

2020 ◽  
Vol 12 (21) ◽  
pp. 3628
Author(s):  
Wei Liang ◽  
Tengfei Zhang ◽  
Wenhui Diao ◽  
Xian Sun ◽  
Liangjin Zhao ◽  
...  
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
Regularization Method ◽  
Training Data ◽  
Image Feature ◽  
Sar Image ◽  
Target Classification ◽  
Spectral Regularization ◽  
Sar Data ◽  
Restriction Problem

Synthetic Aperture Radar (SAR) target classification is an important branch of SAR image interpretation. The deep learning based SAR target classification algorithms have made remarkable achievements. But the acquisition and annotation of SAR target images are time-consuming and laborious, and it is difficult to obtain sufficient training data in many cases. The insufficient training data can make deep learning based models suffering from over-fitting, which will severely limit their wide application in SAR target classification. Motivated by the above problem, this paper employs transfer-learning to transfer the prior knowledge learned from a simulated SAR dataset to a real SAR dataset. To overcome the sample restriction problem caused by the poor feature discriminability for real SAR data. A simple and effective sample spectral regularization method is proposed, which can regularize the singular values of each SAR image feature to improve the feature discriminability. Based on the proposed regularization method, we design a transfer-learning pipeline to leverage the simulated SAR data as well as acquire better feature discriminability. The experimental results indicate that the proposed method is feasible for the sample restriction problem in SAR target classification. Furthermore, the proposed method can improve the classification accuracy when relatively sufficient training data is available, and it can be plugged into any convolutional neural network (CNN) based SAR classification models.

scholarly journals A novel augmented deep transfer learning for classification of COVID-19 and other thoracic diseases from X-rays

2021 ◽  
Author(s):  
Fouzia Altaf ◽  
Syed M. S. Islam ◽  
Naeem Khalid Janjua
Keyword(s):  
Deep Learning ◽  
Medical Imaging ◽  
Transfer Learning ◽  
Medical Images ◽  
Training Data ◽  
Target Domain ◽  
Data Set ◽  
Learning Technique ◽  
Multi Class Classification ◽  
Effective Transfer

AbstractDeep learning has provided numerous breakthroughs in natural imaging tasks. However, its successful application to medical images is severely handicapped with the limited amount of annotated training data. Transfer learning is commonly adopted for the medical imaging tasks. However, a large covariant shift between the source domain of natural images and target domain of medical images results in poor transfer learning. Moreover, scarcity of annotated data for the medical imaging tasks causes further problems for effective transfer learning. To address these problems, we develop an augmented ensemble transfer learning technique that leads to significant performance gain over the conventional transfer learning. Our technique uses an ensemble of deep learning models, where the architecture of each network is modified with extra layers to account for dimensionality change between the images of source and target data domains. Moreover, the model is hierarchically tuned to the target domain with augmented training data. Along with the network ensemble, we also utilize an ensemble of dictionaries that are based on features extracted from the augmented models. The dictionary ensemble provides an additional performance boost to our method. We first establish the effectiveness of our technique with the challenging ChestXray-14 radiography data set. Our experimental results show more than 50% reduction in the error rate with our method as compared to the baseline transfer learning technique. We then apply our technique to a recent COVID-19 data set for binary and multi-class classification tasks. Our technique achieves 99.49% accuracy for the binary classification, and 99.24% for multi-class classification.

scholarly journals A Transfer Learning–Based Active Learning Framework for Brain Tumor Classification

2021 ◽  
Vol 4 ◽  
Author(s):  
Ruqian Hao ◽  
Khashayar Namdar ◽  
Lin Liu ◽  
Farzad Khalvati
Keyword(s):  
Deep Learning ◽  
Brain Tumor ◽  
Active Learning ◽  
Transfer Learning ◽  
Medical Images ◽  
Tumor Classification ◽  
Training Data ◽  
Validation Dataset ◽  
Low Grade ◽  
Learning Framework

Brain tumor is one of the leading causes of cancer-related death globally among children and adults. Precise classification of brain tumor grade (low-grade and high-grade glioma) at an early stage plays a key role in successful prognosis and treatment planning. With recent advances in deep learning, artificial intelligence–enabled brain tumor grading systems can assist radiologists in the interpretation of medical images within seconds. The performance of deep learning techniques is, however, highly depended on the size of the annotated dataset. It is extremely challenging to label a large quantity of medical images, given the complexity and volume of medical data. In this work, we propose a novel transfer learning–based active learning framework to reduce the annotation cost while maintaining stability and robustness of the model performance for brain tumor classification. In this retrospective research, we employed a 2D slice–based approach to train and fine-tune our model on the magnetic resonance imaging (MRI) training dataset of 203 patients and a validation dataset of 66 patients which was used as the baseline. With our proposed method, the model achieved area under receiver operating characteristic (ROC) curve (AUC) of 82.89% on a separate test dataset of 66 patients, which was 2.92% higher than the baseline AUC while saving at least 40% of labeling cost. In order to further examine the robustness of our method, we created a balanced dataset, which underwent the same procedure. The model achieved AUC of 82% compared with AUC of 78.48% for the baseline, which reassures the robustness and stability of our proposed transfer learning augmented with active learning framework while significantly reducing the size of training data.

scholarly journals StyleGANs and Transfer Learning for Generating Synthetic Images in Industrial Applications

Symmetry ◽  
2021 ◽  
Vol 13 (8) ◽  
pp. 1497
Author(s):  
Harold Achicanoy ◽  
Deisy Chaves ◽  
Maria Trujillo
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
Data Augmentation ◽  
Industrial Applications ◽  
Generative Models ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
Augmentation Strategy ◽  
Synthetic Images ◽  
The Impact

Deep learning applications on computer vision involve the use of large-volume and representative data to obtain state-of-the-art results due to the massive number of parameters to optimise in deep models. However, data are limited with asymmetric distributions in industrial applications due to rare cases, legal restrictions, and high image-acquisition costs. Data augmentation based on deep learning generative adversarial networks, such as StyleGAN, has arisen as a way to create training data with symmetric distributions that may improve the generalisation capability of built models. StyleGAN generates highly realistic images in a variety of domains as a data augmentation strategy but requires a large amount of data to build image generators. Thus, transfer learning in conjunction with generative models are used to build models with small datasets. However, there are no reports on the impact of pre-trained generative models, using transfer learning. In this paper, we evaluate a StyleGAN generative model with transfer learning on different application domains—training with paintings, portraits, Pokémon, bedrooms, and cats—to generate target images with different levels of content variability: bean seeds (low variability), faces of subjects between 5 and 19 years old (medium variability), and charcoal (high variability). We used the first version of StyleGAN due to the large number of publicly available pre-trained models. The Fréchet Inception Distance was used for evaluating the quality of synthetic images. We found that StyleGAN with transfer learning produced good quality images, being an alternative for generating realistic synthetic images in the evaluated domains.

scholarly journals Deep learning-based segmentation of lithium-ion battery microstructures enhanced by artificially generated electrodes

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Simon Müller ◽  
Christina Sauter ◽  
Ramesh Shunmugasundaram ◽  
Nils Wenzler ◽  
Vincent De Andrade ◽  
...  
Keyword(s):  
Deep Learning ◽  
Lithium Ion Battery ◽  
Network Performance ◽  
Lithium Ion ◽  
Training Data ◽  
Learning Tools ◽  
Composite Electrodes ◽  
Active Particles ◽  
Volumetric Images ◽  
Learning Data

AbstractAccurate 3D representations of lithium-ion battery electrodes, in which the active particles, binder and pore phases are distinguished and labeled, can assist in understanding and ultimately improving battery performance. Here, we demonstrate a methodology for using deep-learning tools to achieve reliable segmentations of volumetric images of electrodes on which standard segmentation approaches fail due to insufficient contrast. We implement the 3D U-Net architecture for segmentation, and, to overcome the limitations of training data obtained experimentally through imaging, we show how synthetic learning data, consisting of realistic artificial electrode structures and their tomographic reconstructions, can be generated and used to enhance network performance. We apply our method to segment x-ray tomographic microscopy images of graphite-silicon composite electrodes and show it is accurate across standard metrics. We then apply it to obtain a statistically meaningful analysis of the microstructural evolution of the carbon-black and binder domain during battery operation.

scholarly journals Transfer Learning for Inference of Metastatic Origin from Whole Slide Histology

2021 ◽  
Author(s):  
Geoffrey F. Schau ◽  
Hassan Ghani ◽  
Erik A. Burlingame ◽  
Guillaume Thibault ◽  
Joe W. Gray ◽  
...  
Keyword(s):  
Deep Learning ◽  
Transfer Learning ◽  
Primary Tumor ◽  
Large Scale ◽  
Metastatic Cancer ◽  
Clinical Diagnostics ◽  
Training Data ◽  
Fine Tuning ◽  
Learning Approach ◽  
Whole Slide Images

AbstractAccurate diagnosis of metastatic cancer is essential for prescribing optimal control strategies to halt further spread of metastasizing disease. While pathological inspection aided by immunohistochemistry staining provides a valuable gold standard for clinical diagnostics, deep learning methods have emerged as powerful tools for identifying clinically relevant features of whole slide histology relevant to a tumor’s metastatic origin. Although deep learning models require significant training data to learn effectively, transfer learning paradigms provide mechanisms to circumvent limited training data by first training a model on related data prior to fine-tuning on smaller data sets of interest. In this work we propose a transfer learning approach that trains a convolutional neural network to infer the metastatic origin of tumor tissue from whole slide images of hematoxylin and eosin (H&E) stained tissue sections and illustrate the advantages of pre-training network on whole slide images of primary tumor morphology. We further characterize statistical dissimilarity between primary and metastatic tumors of various indications on patch-level images to highlight limitations of our indication-specific transfer learning approach. Using a primary-to-metastatic transfer learning approach, we achieved mean class-specific areas under receiver operator characteristics curve (AUROC) of 0.779, which outperformed comparable models trained on only images of primary tumor (mean AUROC of 0.691) or trained on only images of metastatic tumor (mean AUROC of 0.675), supporting the use of large scale primary tumor imaging data in developing computer vision models to characterize metastatic origin of tumor lesions.

Sign in / Sign up

Export Citation Format

Share Document