scholarly journals A Tensor Space Model-Based Deep Neural Network for Text Classification

2021 ◽  
Vol 11 (20) ◽  
pp. 9703
Author(s):  
Han-joon Kim ◽  
Pureum Lim
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Text Classification ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Classification Systems ◽  
Support Vector ◽  
Tensor Space ◽  
Space Model

Most text classification systems use machine learning algorithms; among these, naïve Bayes and support vector machine algorithms adapted to handle text data afford reasonable performance. Recently, given developments in deep learning technology, several scholars have used deep neural networks (recurrent and convolutional neural networks) to improve text classification. However, deep learning-based text classification has not greatly improved performance compared to that of conventional algorithms. This is because a textual document is essentially expressed as a vector (only), albeit with word dimensions, which compromises the inherent semantic information, even if the vector is (appropriately) transformed to add conceptual information. To solve this `loss of term senses’ problem, we develop a concept-driven deep neural network based upon our semantic tensor space model. The semantic tensor used for text representation features a dependency between the term and the concept; we use this to develop three deep neural networks for text classification. We perform experiments using three standard document corpora, and we show that our proposed methods are superior to both traditional and more recent learning methods.

scholarly journals A deep neural network approach to predicting clinical outcomes of neuroblastoma patients

2019 ◽  
Author(s):  
Léon-Charles Tranchevent ◽  
Francisco Azuaje ◽  
Jagath C. Rajapakse
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Clinical Outcomes ◽  
Deep Neural Network ◽  
Superior Performance ◽  
Support Vector ◽  
Disease Etiology ◽  
Topological Features ◽  
Wide Range

AbstractThe availability of high-throughput omics datasets from large patient cohorts has allowed the development of methods that aim at predicting patient clinical outcomes, such as survival and disease recurrence. Such methods are also important to better understand the biological mechanisms underlying disease etiology and development, as well as treatment responses. Recently, different predictive models, relying on distinct algorithms (including Support Vector Machines and Random Forests) have been investigated. In this context, deep learning strategies are of special interest due to their demonstrated superior performance over a wide range of problems and datasets. One of the main challenges of such strategies is the “small n large p” problem. Indeed, omics datasets typically consist of small numbers of samples and large numbers of features relative to typical deep learning datasets. Neural networks usually tackle this problem through feature selection or by including additional constraints during the learning process.We propose to tackle this problem with a novel strategy that relies on a graph-based method for feature extraction, coupled with a deep neural network for clinical outcome prediction. The omics data are first represented as graphs whose nodes represent patients, and edges represent correlations between the patients’ omics profiles. Topological features, such as centralities, are then extracted from these graphs for every node. Lastly, these features are used as input to train and test various classifiers.We apply this strategy to four neuroblastoma datasets and observe that models based on neural networks are more accurate than state of the art models (DNN: 85%-87%, SVM/RF: 75%-82%). We explore how different parameters and configurations are selected in order to overcome the effects of the small data problem as well as the curse of dimensionality. Our results indicate that the deep neural networks capture complex features in the data that help predicting patient clinical outcomes.

scholarly journals Tri-net for Semi-Supervised Deep Learning

2018 ◽  
Author(s):  
Dong-Dong Chen ◽  
Wei Wang ◽  
Wei Gao ◽  
Zhi-Hua Zhou
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Error Rate ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Fine Tuning ◽  
Learning Methods ◽  
Model Initialization

Deep neural networks have witnessed great successes in various real applications, but it requires a large number of labeled data for training. In this paper, we propose tri-net, a deep neural network which is able to use massive unlabeled data to help learning with limited labeled data. We consider model initialization, diversity augmentation and pseudo-label editing simultaneously. In our work, we utilize output smearing to initialize modules, use fine-tuning on labeled data to augment diversity and eliminate unstable pseudo-labels to alleviate the influence of suspicious pseudo-labeled data. Experiments show that our method achieves the best performance in comparison with state-of-the-art semi-supervised deep learning methods. In particular, it achieves 8.30% error rate on CIFAR-10 by using only 4000 labeled examples.

scholarly journals A deep neural network approach to predicting clinical outcomes of neuroblastoma patients

2019 ◽  
Vol 12 (S8) ◽  
Author(s):  
Léon-Charles Tranchevent ◽  
Francisco Azuaje ◽  
Jagath C. Rajapakse
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Clinical Outcomes ◽  
Deep Neural Network ◽  
Superior Performance ◽  
Support Vector ◽  
Disease Etiology ◽  
Topological Features ◽  
Wide Range

Abstract Background The availability of high-throughput omics datasets from large patient cohorts has allowed the development of methods that aim at predicting patient clinical outcomes, such as survival and disease recurrence. Such methods are also important to better understand the biological mechanisms underlying disease etiology and development, as well as treatment responses. Recently, different predictive models, relying on distinct algorithms (including Support Vector Machines and Random Forests) have been investigated. In this context, deep learning strategies are of special interest due to their demonstrated superior performance over a wide range of problems and datasets. One of the main challenges of such strategies is the “small n large p” problem. Indeed, omics datasets typically consist of small numbers of samples and large numbers of features relative to typical deep learning datasets. Neural networks usually tackle this problem through feature selection or by including additional constraints during the learning process. Methods We propose to tackle this problem with a novel strategy that relies on a graph-based method for feature extraction, coupled with a deep neural network for clinical outcome prediction. The omics data are first represented as graphs whose nodes represent patients, and edges represent correlations between the patients’ omics profiles. Topological features, such as centralities, are then extracted from these graphs for every node. Lastly, these features are used as input to train and test various classifiers. Results We apply this strategy to four neuroblastoma datasets and observe that models based on neural networks are more accurate than state of the art models (DNN: 85%-87%, SVM/RF: 75%-82%). We explore how different parameters and configurations are selected in order to overcome the effects of the small data problem as well as the curse of dimensionality. Conclusions Our results indicate that the deep neural networks capture complex features in the data that help predicting patient clinical outcomes.

scholarly journals Research on Time Series Anomaly Detection: Based on Deep Learning Methods

2021 ◽  
Vol 2132 (1) ◽  
pp. 012012
Author(s):  
Jiaqi Zhou
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Time Series ◽  
Deep Learning ◽  
Anomaly Detection ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Detection Task ◽  
Detection Methods ◽  
Learning Methods

Abstract Time series anomaly detection has always been an important research direction. The early time series anomaly detection methods are mainly statistical methods and machine learning methods. With the powerful functions of deep neural network being continuously mined by researchers, the effect of deep neural network in anomaly detection task has been significantly better than the traditional methods. In view of the continuous development and application of deep neural networks such as transformer and graph neural network (GNN) in time series anomaly detection in recent years, the body of research lacks a comparative evaluation of deep learning methods in recent years. This paper studies various deep neural networks suitable for time series, which are divided into three categories according to anomaly detection methods. The evaluation is conducted on public datasets. By analyzing the evaluation criteria, this paper discusses the performance of each model, as well as the problems and development direction in the field of time series anomaly detection in the future. This study found that in the time series anomaly detection task, transformer is suitable for dealing with long-time series prediction, and studying the graph structure of time series may be the best way to deal with time series anomaly detection in the future

Deep distributed convolutional neural networks: Universality

2018 ◽  
Vol 16 (06) ◽  
pp. 895-919 ◽  
Author(s):  
Ding-Xuan Zhou
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Convolutional Neural Networks ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Mathematical Foundation ◽  
Deep Convolutional Neural Networks ◽  
The Family ◽  
Fully Connected

Deep learning based on structured deep neural networks has provided powerful applications in various fields. The structures imposed on the deep neural networks are crucial, which makes deep learning essentially different from classical schemes based on fully connected neural networks. One of the commonly used deep neural network structures is generated by convolutions. The produced deep learning algorithms form the family of deep convolutional neural networks. Despite of their power in some practical domains, little is known about the mathematical foundation of deep convolutional neural networks such as universality of approximation. In this paper, we propose a family of new structured deep neural networks: deep distributed convolutional neural networks. We show that these deep neural networks have the same order of computational complexity as the deep convolutional neural networks, and we prove their universality of approximation. Some ideas of our analysis are from ridge approximation, wavelets, and learning theory.

scholarly journals Deep anomaly detection for industrial systems: a case study

2020 ◽  
Vol 12 (1) ◽  
pp. 8
Author(s):  
Feng Xue ◽  
Weizhong Yan ◽  
Tianyi Wang ◽  
Hao Huang ◽  
Bojun Feng
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Time Series ◽  
Anomaly Detection ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Normal Operation ◽  
Series Data ◽  
Support Vector ◽  
Industrial Systems

We explore the use of deep neural networks for anomaly detection of industrial systems where the data are multivariate time series measurements. We formulate the problem as a self-supervised learning where data under normal operation is used to train a deep neural network autoregressive model, i.e., use a window of time series data to predict future data values. The aim of such a model is to learn to represent the system dynamic behavior under normal conditions, while expect higher model vs. measurement discrepancies under faulty conditions. In real world applications, many control settings are categorical in nature. In this paper, vector embedding and joint losses are employed to deal with such situations. Both LSTM and CNN based deep neural network backbones are studied on the Secure Water Treatment (SWaT) testbed datasets. Also, Support Vector Data Description (SVDD) method is adapted to such anomaly detection settings with deep neural networks. Evaluation methods and results are discussed based on the SWaT dataset along with potential pitfalls.

Teeth and Landmarks Detection and Classification Based on Deep Neural Networks

2019 ◽  
pp. 129-150
Author(s):  
Lyudmila N. Tuzova ◽  
Dmitry V. Tuzoff ◽  
Sergey I. Nikolenko ◽  
Alexey S. Krasnov
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Image Analysis ◽  
Deep Learning ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Rapid Development ◽  
Image Interpretation ◽  
Semantic Segmentation ◽  
Recent Decade

In the recent decade, deep neural networks have enjoyed rapid development in various domains, including medicine. Convolutional neural networks (CNNs), deep neural network structures commonly used for image interpretation, brought the breakthrough in computer vision and became state-of-the-art techniques for various image recognition tasks, such as image classification, object detection, and semantic segmentation. In this chapter, the authors provide an overview of deep learning algorithms and review available literature for dental image analysis with methods based on CNNs. The present study is focused on the problems of landmarks and teeth detection and classification, as these tasks comprise an essential part of dental image interpretation both in clinical dentistry and in human identification systems based on the dental biometrical information.

scholarly journals Where Does Trust Break Down? A Quantitative Trust Analysis of Deep Neural Networks via Trust Matrix and Conditional Trust Densities

2021 ◽  
Vol 6 (1) ◽  
pp. 1-5
Author(s):  
Andrew Hryniowski ◽  
Alexander Wong ◽  
Xiao Yu Wang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Trust Metric

With tremendous rise in deep learning adoption comes questions about the trustworthiness of the deep neural networks that power a variety of applications. In this work, we introduce the concept of trust matrix, a novel trust quantification strategy that leverages the recently introduced question-answer trust metric by Wong et al. to provide deeper, more detailed insights into where trust breaks down for a given deep neural network given a set of questions. More specifically, a trust matrix defines the expected question-answer trust for a given actor-oracle answer scenario, allowing one to quickly spot areas of low trust that needs to be addressed in order to improve the trustworthiness of a deep neural network. We further extend the concept of trust densities with the notion of conditional trust densities.

scholarly journals Deep neural networks using a single neuron: folded-in-time architecture using feedback-modulated delay loops

2021 ◽  
Vol 12 (1) ◽  
Author(s):  
Florian Stelzer ◽  
André Röhm ◽  
Raul Vicente ◽  
Ingo Fischer ◽  
Serhiy Yanchuk
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Neural Network ◽  
Single Neuron ◽  
Deep Neural Networks ◽  
Back Propagation ◽  
Local Network ◽  
Multiple Time ◽  
Learning Tools ◽  
Back Propagation Algorithm

AbstractDeep neural networks are among the most widely applied machine learning tools showing outstanding performance in a broad range of tasks. We present a method for folding a deep neural network of arbitrary size into a single neuron with multiple time-delayed feedback loops. This single-neuron deep neural network comprises only a single nonlinearity and appropriately adjusted modulations of the feedback signals. The network states emerge in time as a temporal unfolding of the neuron’s dynamics. By adjusting the feedback-modulation within the loops, we adapt the network’s connection weights. These connection weights are determined via a back-propagation algorithm, where both the delay-induced and local network connections must be taken into account. Our approach can fully represent standard Deep Neural Networks (DNN), encompasses sparse DNNs, and extends the DNN concept toward dynamical systems implementations. The new method, which we call Folded-in-time DNN (Fit-DNN), exhibits promising performance in a set of benchmark tasks.

Inclusion of Multiple Cycling of the Potential into Deep Neural Network Classification of Voltammetric Reaction Mechanisms

2021 ◽  
Author(s):  
Luke Gundry ◽  
Gareth Kennedy ◽  
Alan Bond ◽  
Jie Zhang
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Reaction Mechanisms ◽  
Deep Neural Network ◽  
Deep Neural Networks ◽  
Neural Network Classification ◽  
Initial Cycle

The use of Deep Neural Networks (DNNs) for the classification of electrochemical mechanisms based on training with simulations of the initial cycle of potential have been reported. In this paper,...

Sign in / Sign up

Export Citation Format

Share Document