Feature extraction for deep neural networks based on decision boundaries

2017 ◽  
Author(s):  
Seongyoun Woo ◽  
Chulhee Lee
Keyword(s):  
Neural Networks ◽  
Feature Extraction ◽  
Deep Neural Networks ◽  
Decision Boundaries

scholarly journals Modular Dynamic Neural Network: A Continual Learning Architecture

2021 ◽  
Vol 11 (24) ◽  
pp. 12078
Author(s):  
Daniel Turner ◽  
Pedro J. S. Cardoso ◽  
João M. F. Rodrigues
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Feature Extraction ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Simple Task ◽  
Dynamic Neural Network ◽  
Main Components ◽  
Over Time ◽  
Continual Learning

Learning to recognize a new object after having learned to recognize other objects may be a simple task for a human, but not for machines. The present go-to approaches for teaching a machine to recognize a set of objects are based on the use of deep neural networks (DNN). So, intuitively, the solution for teaching new objects on the fly to a machine should be DNN. The problem is that the trained DNN weights used to classify the initial set of objects are extremely fragile, meaning that any change to those weights can severely damage the capacity to perform the initial recognitions; this phenomenon is known as catastrophic forgetting (CF). This paper presents a new (DNN) continual learning (CL) architecture that can deal with CF, the modular dynamic neural network (MDNN). The presented architecture consists of two main components: (a) the ResNet50-based feature extraction component as the backbone; and (b) the modular dynamic classification component, which consists of multiple sub-networks and progressively builds itself up in a tree-like structure that rearranges itself as it learns over time in such a way that each sub-network can function independently. The main contribution of the paper is a new architecture that is strongly based on its modular dynamic training feature. This modular structure allows for new classes to be added while only altering specific sub-networks in such a way that previously known classes are not forgotten. Tests on the CORe50 dataset showed results above the state of the art for CL architectures.

Bearing fault diagnosis using deep learning techniques coupled with handcrafted feature extraction: A comparative study

2020 ◽  
pp. 107754632092914
Author(s):  
Mohammed Alabsi ◽  
Yabin Liao ◽  
Ala-Addin Nabulsi
Keyword(s):  
Neural Networks ◽  
Feature Extraction ◽  
Deep Learning ◽  
Comparative Study ◽  
Domain Knowledge ◽  
Deep Neural Networks ◽  
Performance Limits ◽  
Data Repositories ◽  
Learning Techniques ◽  
Wide Range

Deep learning has seen tremendous growth over the past decade. It has set new performance limits for a wide range of applications, including computer vision, speech recognition, and machinery health monitoring. With the abundance of instrumentation data and the availability of high computational power, deep learning continues to prove itself as an efficient tool for the extraction of micropatterns from machinery big data repositories. This study presents a comparative study for feature extraction capabilities using stacked autoencoders considering the use of expert domain knowledge. Case Western Reserve University bearing dataset was used for the study, and a classifier was trained and tested to extract and visualize features from 12 different failure classes. Based on the raw data preprocessing, four different deep neural network structures were studied. Results indicated that integrating domain knowledge with deep learning techniques improved feature extraction capabilities and reduced the deep neural networks size and computational requirements without the need for exhaustive deep neural networks architecture tuning and modification.

scholarly journals An Automated ECG Beat Classification System Using Deep Neural Networks with an Unsupervised Feature Extraction Technique

2019 ◽  
Vol 9 (14) ◽  
pp. 2921 ◽  
Author(s):  
Siti Nurmaini ◽  
Radiyati Umi Partan ◽  
Wahyu Caesarendra ◽  
Tresna Dewi ◽  
Muhammad Naufal Rahmatullah ◽  
...  
Keyword(s):  
Machine Learning ◽  
Neural Networks ◽  
Feature Extraction ◽  
Feature Selection ◽  
Deep Neural Networks ◽  
Machine Learning Technique ◽  
Proposed Model ◽  
Learning Technique ◽  
Processing Steps ◽  
Unsupervised Feature Extraction

An automated classification system based on a Deep Learning (DL) technique for Cardiac Disease (CD) monitoring and detection is proposed in this paper. The proposed DL architecture is divided into Deep Auto-Encoders (DAEs) as an unsupervised form of feature learning and Deep Neural Networks (DNNs) as a classifier. The objective of this study is to improve on the previous machine learning technique that consists of several data processing steps such as feature extraction and feature selection or feature reduction. It is also noticed that the previously used machine learning technique required human interference and expertise in determining robust features, yet was time-consuming in the labeling and data processing steps. In contrast, DL enables an embedded feature extraction and feature selection in DAEs pre-training and DNNs fine-tuning process directly from raw data. Hence, DAEs is able to extract high-level of features not only from the training data but also from unseen data. The proposed model uses 10 classes of imbalanced data from ECG signals. Since it is related to the cardiac region, abnormality is usually considered for an early diagnosis of CD. In order to validate the result, the proposed model is compared with the shallow models and DL approaches. Results found that the proposed method achieved a promising performance with 99.73% accuracy, 91.20% sensitivity, 93.60% precision, 99.80% specificity, and a 91.80% F1-Score. Moreover, both the Receiver Operating Characteristic (ROC) curve and the Precision-Recall (PR) curve from the confusion matrix showed that the developed model is a good classifier. The developed model based on unsupervised feature extraction and deep neural network is ready to be used on a large population before its installation for clinical usage.

scholarly journals Multimodal Feature Extraction and Fusion Deep Neural Networks for Short-Term Load Forecasting

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 185373-185383
Author(s):  
Zhengmin Kong ◽  
Chenggang Zhang ◽  
He Lv ◽  
Feng Xiong ◽  
Zhuolin Fu
Keyword(s):  
Neural Networks ◽  
Feature Extraction ◽  
Deep Neural Networks ◽  
Load Forecasting ◽  
Short Term ◽  
Short Term Load Forecasting
Sign in / Sign up

Export Citation Format

Share Document