scholarly journals DEEPCON: Protein Contact Prediction using Dilated Convolutional Neural Networks with Dropout

2019 ◽  
Author(s):  
Badri Adhikari
Keyword(s):  
Neural Networks ◽  
Long Range ◽  
High Throughput Sequencing ◽  
State Of The Art ◽  
Contact Prediction ◽  
Homologous Sequences ◽  
Learning Techniques ◽  
Network Methods ◽  
Medium Range ◽  
Art Methods

AbstractBackgroundExciting new opportunities have arisen to solve the protein contact prediction problem from the progress in neural networks and the availability of a large number of homologous sequences through high-throughput sequencing. In this work, we study how deep convolutional neural network methods (ConvNets) may be best designed and developed to solve this long-standing problem.MethodWith publicly available datasets, we designed and trained various ConvNet architectures. We tested several recent deep learning techniques including wide residual networks, dropouts, and dilated convolutions. We studied the improvements in the precision of medium-range and long-range contacts, and compared the performance of our best architectures with the ones used in existing state-of-the-art methods.ResultsThe proposed ConvNet architectures predict contacts with significantly more precision than the architectures used in several state-of-the-art methods. When trained using the DeepCov dataset consisting of 3,456 proteins and tested on PSICOV dataset of 150 proteins, our architectures achieve up to 15% higher precision when L/2 long-range contacts are evaluated. Similarly, when trained using the DNCON2 dataset consisting of 1,426 proteins and tested on 84 protein domains in the CASP12 dataset, our single network achieves 4.8% higher precision than the ensembled DNCON2 method when top L long-range contacts are evaluated. DEEPCON will be made publicly available athttps://github.com/badriadhikari/DEEPCON/.

scholarly journals DEEPCON: protein contact prediction using dilated convolutional neural networks with dropout

2019 ◽  
Vol 36 (2) ◽  
pp. 470-477 ◽  
Author(s):  
Badri Adhikari
Keyword(s):  
Neural Networks ◽  
Long Range ◽  
Convolutional Neural Networks ◽  
High Throughput Sequencing ◽  
State Of The Art ◽  
Deep Convolutional Neural Networks ◽  
Contact Prediction ◽  
Learning Techniques ◽  
Medium Range ◽  
Art Methods

Abstract Motivation Exciting new opportunities have arisen to solve the protein contact prediction problem from the progress in neural networks and the availability of a large number of homologous sequences through high-throughput sequencing. In this work, we study how deep convolutional neural networks (ConvNets) may be best designed and developed to solve this long-standing problem. Results With publicly available datasets, we designed and trained various ConvNet architectures. We tested several recent deep learning techniques including wide residual networks, dropouts and dilated convolutions. We studied the improvements in the precision of medium-range and long-range contacts, and compared the performance of our best architectures with the ones used in existing state-of-the-art methods. The proposed ConvNet architectures predict contacts with significantly more precision than the architectures used in several state-of-the-art methods. When trained using the DeepCov dataset consisting of 3456 proteins and tested on PSICOV dataset of 150 proteins, our architectures achieve up to 15% higher precision when L/2 long-range contacts are evaluated. Similarly, when trained using the DNCON2 dataset consisting of 1426 proteins and tested on 84 protein domains in the CASP12 dataset, our single network achieves 4.8% higher precision than the ensembled DNCON2 method when top L long-range contacts are evaluated. Availability and implementation DEEPCON is available at https://github.com/badriadhikari/DEEPCON/.

scholarly journals Uncertainty-Based Human-in-the-Loop Deep Learning for Land Cover Segmentation

2020 ◽  
Vol 12 (22) ◽  
pp. 3836
Author(s):  
Carlos García Rodríguez ◽  
Jordi Vitrià ◽  
Oscar Mora
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Land Cover ◽  
Satellite Images ◽  
State Of The Art ◽  
Low Cost ◽  
Human Intervention ◽  
Human In The Loop ◽  
Learning Techniques

In recent years, different deep learning techniques were applied to segment aerial and satellite images. Nevertheless, state of the art techniques for land cover segmentation does not provide accurate results to be used in real applications. This is a problem faced by institutions and companies that want to replace time-consuming and exhausting human work with AI technology. In this work, we propose a method that combines deep learning with a human-in-the-loop strategy to achieve expert-level results at a low cost. We use a neural network to segment the images. In parallel, another network is used to measure uncertainty for predicted pixels. Finally, we combine these neural networks with a human-in-the-loop approach to produce correct predictions as if developed by human photointerpreters. Applying this methodology shows that we can increase the accuracy of land cover segmentation tasks while decreasing human intervention.

scholarly journals CapsTM: capsule network for Chinese medical text matching

2021 ◽  
Vol 21 (S2) ◽  
Author(s):  
Xiaoming Yu ◽  
Yedan Shen ◽  
Yuan Ni ◽  
Xiaowei Huang ◽  
Xiaolong Wang ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Deep Learning ◽  
Network Architecture ◽  
State Of The Art ◽  
Interaction Matrix ◽  
Application Systems ◽  
Input Layer ◽  
Art Methods ◽  
Text Matching

Abstract Background Text Matching (TM) is a fundamental task of natural language processing widely used in many application systems such as information retrieval, automatic question answering, machine translation, dialogue system, reading comprehension, etc. In recent years, a large number of deep learning neural networks have been applied to TM, and have refreshed benchmarks of TM repeatedly. Among the deep learning neural networks, convolutional neural network (CNN) is one of the most popular networks, which suffers from difficulties in dealing with small samples and keeping relative structures of features. In this paper, we propose a novel deep learning architecture based on capsule network for TM, called CapsTM, where capsule network is a new type of neural network architecture proposed to address some of the short comings of CNN and shows great potential in many tasks. Methods CapsTM is a five-layer neural network, including an input layer, a representation layer, an aggregation layer, a capsule layer and a prediction layer. In CapsTM, two pieces of text are first individually converted into sequences of embeddings and are further transformed by a highway network in the input layer. Then, Bidirectional Long Short-Term Memory (BiLSTM) is used to represent each piece of text and attention-based interaction matrix is used to represent interactive information of the two pieces of text in the representation layer. Subsequently, the two kinds of representations are fused together by BiLSTM in the aggregation layer, and are further represented with capsules (vectors) in the capsule layer. Finally, the prediction layer is a connected network used for classification. CapsTM is an extension of ESIM by adding a capsule layer before the prediction layer. Results We construct a corpus of Chinese medical question matching, which contains 36,360 question pairs. This corpus is randomly split into three parts: a training set of 32,360 question pairs, a development set of 2000 question pairs and a test set of 2000 question pairs. On this corpus, we conduct a series of experiments to evaluate the proposed CapsTM and compare it with other state-of-the-art methods. CapsTM achieves the highest F-score of 0.8666. Conclusion The experimental results demonstrate that CapsTM is effective for Chinese medical question matching and outperforms other state-of-the-art methods for comparison.

scholarly journals Graph Attention Based Proposal 3D ConvNets for Action Detection

2020 ◽  
Vol 34 (04) ◽  
pp. 4626-4633 ◽  
Author(s):  
Jin Li ◽  
Xianglong Liu ◽  
Zhuofan Zong ◽  
Wanru Zhao ◽  
Mingyuan Zhang ◽  
...  
Keyword(s):  
Neural Networks ◽  
Long Range ◽  
Convolutional Neural Networks ◽  
Adjacency Matrix ◽  
State Of The Art ◽  
Original Model ◽  
Action Detection ◽  
Model Experiments ◽  
Temporal And Spatial ◽  
Temporal Action

The recent advances in 3D Convolutional Neural Networks (3D CNNs) have shown promising performance for untrimmed video action detection, employing the popular detection framework that heavily relies on the temporal action proposal generations as the input of the action detector and localization regressor. In practice the proposals usually contain strong intra and inter relations among them, mainly stemming from the temporal and spatial variations in the video actions. However, most of existing 3D CNNs ignore the relations and thus suffer from the redundant proposals degenerating the detection performance and efficiency. To address this problem, we propose graph attention based proposal 3D ConvNets (AGCN-P-3DCNNs) for video action detection. Specifically, our proposed graph attention is composed of intra attention based GCN and inter attention based GCN. We use intra attention to learn the intra long-range dependencies inside each action proposal and update node matrix of Intra Attention based GCN, and use inter attention to learn the inter dependencies between different action proposals as adjacency matrix of Inter Attention based GCN. Afterwards, we fuse intra and inter attention to model intra long-range dependencies and inter dependencies simultaneously. Another contribution is that we propose a simple and effective framewise classifier, which enhances the feature presentation capabilities of backbone model. Experiments on two proposal 3D ConvNets based models (P-C3D and P-ResNet) and two popular action detection benchmarks (THUMOS 2014, ActivityNet v1.3) demonstrate the state-of-the-art performance achieved by our method. Particularly, P-C3D embedded with our module achieves average mAP 3.7% improvement on THUMOS 2014 dataset compared to original model.

scholarly journals A Survey of Handwritten Character Recognition with MNIST and EMNIST

2019 ◽  
Vol 9 (15) ◽  
pp. 3169 ◽  
Author(s):  
Alejandro Baldominos ◽  
Yago Saez ◽  
Pedro Isasi
Keyword(s):  
Neural Networks ◽  
Character Recognition ◽  
Data Augmentation ◽  
State Of The Art ◽  
Handwritten Digit Recognition ◽  
Test Error ◽  
Digit Recognition ◽  
Original Dataset ◽  
Learning Techniques ◽  
Handwritten Digit

This paper summarizes the top state-of-the-art contributions reported on the MNIST dataset for handwritten digit recognition. This dataset has been extensively used to validate novel techniques in computer vision, and in recent years, many authors have explored the performance of convolutional neural networks (CNNs) and other deep learning techniques over this dataset. To the best of our knowledge, this paper is the first exhaustive and updated review of this dataset; there are some online rankings, but they are outdated, and most published papers survey only closely related works, omitting most of the literature. This paper makes a distinction between those works using some kind of data augmentation and works using the original dataset out-of-the-box. Also, works using CNNs are reported separately; as they are becoming the state-of-the-art approach for solving this problem. Nowadays, a significant amount of works have attained a test error rate smaller than 1% on this dataset; which is becoming non-challenging. By mid-2017, a new dataset was introduced: EMNIST, which involves both digits and letters, with a larger amount of data acquired from a database different than MNIST’s. In this paper, EMNIST is explained and some results are surveyed.

scholarly journals Person Re-Identification Using Convolutional Neural Network and Autoencoder Embedded on Frameworks Based on Siamese and Triplet Networks

2020 ◽  
Author(s):  
Fábia Isabella Pires Enembreck ◽  
Erikson Freitas de Morais ◽  
Marcella Scoczynski Ribeiro Martins
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Identification Problem ◽  
Reference Image ◽  
Learning Techniques ◽  
Art Methods

Abstract The person re-identification problem addresses the task of identify if a person being watched by security cameras in surveillance environments has ever been in the scene. This problem is considered challenging, since the images obtained by cameras are subject to many variations, such as lighting, perspective and occlusions. This work aims to develop two robust approaches based on deep learning techniques for person re-identification, considering these variations. The first approach uses a Siamese neural network composed by two identical subnets. This model receives two input images that may or may not be from the same person. The second approach consists of a triplet neural network, with three identical subnets, which receives a reference image from a certain person, a second image from the same person and another image from a different person. Both approaches have identical subnets, composed by a convolutional neural network which extracts general characteristics from each image and an autoencoder model, responsible for addressing high variations that input images may undergo. To compare the developed networks, three datasets were used, and the accuracy and the CMC curve metrics were applied for the analysis. The experiments showed an improvement in the results with the use of the autoencoder in the subnets. Besides, Triplet Neural Network presented promising results in comparison with Siamese Neural Network and state-of-the-art methods.

scholarly journals ECG Denoising Using Artificial Neural Networks and Complete Ensemble Empirical Mode Decomposition

2021 ◽  
Vol 12 (2) ◽  
pp. 2382-2389
Author(s):  
Rajesh Birok, Et. al.
Keyword(s):  
Neural Networks ◽  
Empirical Mode Decomposition ◽  
State Of The Art ◽  
Signal To Noise Ratio ◽  
Low Frequency ◽  
Ensemble Empirical Mode Decomposition ◽  
Novel Approach ◽  
Mode Decomposition ◽  
The Neural Network ◽  
Art Methods

Electrocardiogram (ECG) is a documentation of the electrical activities of the heart. It is used to identify a number of cardiac faults such as arrhythmias, AF etc.  Quite often the ECG gets corrupted by various kinds of artifacts, thus in order to gain correct information from them, they must first be denoised. This paper presents a novel approach for the filtering of low frequency artifacts of ECG signals by using Complete Ensemble Empirical Mode Decomposition (CEED) and Neural Networks, which removes most of the constituent noise while assuring no loss of information in terms of the morphology of the ECG signal. The contribution of the method lies in the fact that it combines the advantages of both EEMD and ANN. The use of CEEMD ensures that the Neural Network does not get over fitted. It also significantly helps in building better predictors at individual frequency levels. The proposed method is compared with other state-of-the-art methods in terms of Mean Square Error (MSE), Signal to Noise Ratio (SNR) and Correlation Coefficient. The results show that the proposed method has better performance as compared to other state-of-the-art methods for low frequency artifacts removal from EEG.  

Graph Deep Learning for Long Range Forecasting

2021 ◽  
Author(s):  
Salva Rühling Cachay ◽  
Emma Erickson ◽  
Arthur Fender C. Bucker ◽  
Ernest Pokropek ◽  
Willa Potosnak ◽  
...  
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
Long Range ◽  
Large Scale ◽  
State Of The Art ◽  
Southern Oscillation ◽  
Predictive Skill ◽  
Anomaly Pattern ◽  
Previous State ◽  
Graph Neural Networks

<p>Deep learning-based models have been recently shown to be competitive with, or even outperform, state-of-the-art long range forecasting models, such as for projecting the El Niño-Southern Oscillation (ENSO). However, current deep learning models are based on convolutional neural networks which are difficult to interpret and can fail to model large-scale dependencies, such as teleconnections, that are particularly important for long range projections. Hence, we propose to explicitly model large-scale dependencies with Graph Neural Networks (GNN) to enhance explainability and improve the predictive skill of long lead time forecasts.</p><p>In preliminary experiments focusing on ENSO, our GNN model outperforms previous state-of-the-art machine learning based systems for forecasts up to 6 months ahead. The explicit modeling of information flow via edges makes our model more explainable, and it is indeed shown to learn a sensible graph structure from scratch that correlates with the ENSO anomaly pattern for a given number of lead months.</p><p> </p>

scholarly journals Methodology for Efficient CNN Architectures in Profiling Attacks

2019 ◽  
pp. 1-36
Author(s):  
Gabriel Zaid ◽  
Lilian Bossuet ◽  
Amaury Habrard ◽  
Alexandre Venelli
Keyword(s):  
Neural Networks ◽  
Deep Learning ◽  
State Of The Art ◽  
Side Channel ◽  
Clear Understanding ◽  
Learning Techniques ◽  
Previous State ◽  
Template Attacks ◽  
Public Datasets ◽  
Visualization Techniques

The side-channel community recently investigated a new approach, based on deep learning, to significantly improve profiled attacks against embedded systems. Previous works have shown the benefit of using convolutional neural networks (CNN) to limit the effect of some countermeasures such as desynchronization. Compared with template attacks, deep learning techniques can deal with trace misalignment and the high dimensionality of the data. Pre-processing is no longer mandatory. However, the performance of attacks depends to a great extent on the choice of each hyperparameter used to configure a CNN architecture. Hence, we cannot perfectly harness the potential of deep neural networks without a clear understanding of the network’s inner-workings. To reduce this gap, we propose to clearly explain the role of each hyperparameters during the feature selection phase using some specific visualization techniques including Weight Visualization, Gradient Visualization and Heatmaps. By highlighting which features are retained by filters, heatmaps come in handy when a security evaluator tries to interpret and understand the efficiency of CNN. We propose a methodology for building efficient CNN architectures in terms of attack efficiency and network complexity, even in the presence of desynchronization. We evaluate our methodology using public datasets with and without desynchronization. In each case, our methodology outperforms the previous state-of-the-art CNN models while significantly reducing network complexity. Our networks are up to 25 times more efficient than previous state-of-the-art while their complexity is up to 31810 times smaller. Our results show that CNN networks do not need to be very complex to perform well in the side-channel context.

scholarly journals Accurate Temporal Action Proposal Generation with Relation-Aware Pyramid Network

2020 ◽  
Vol 34 (07) ◽  
pp. 10810-10817
Author(s):  
Jialin Gao ◽  
Zhixiang Shi ◽  
Guanshuo Wang ◽  
Jiani Li ◽  
Yufeng Yuan ◽  
...  
Keyword(s):  
Long Range ◽  
State Of The Art ◽  
Contextual Information ◽  
Local Features ◽  
Two Stage ◽  
Art Methods ◽  
Adjustment Scheme ◽  
Temporal Action

Accurate temporal action proposals play an important role in detecting actions from untrimmed videos. The existing approaches have difficulties in capturing global contextual information and simultaneously localizing actions with different durations. To this end, we propose a Relation-aware pyramid Network (RapNet) to generate highly accurate temporal action proposals. In RapNet, a novel relation-aware module is introduced to exploit bi-directional long-range relations between local features for context distilling. This embedded module enhances the RapNet in terms of its multi-granularity temporal proposal generation ability, given predefined anchor boxes. We further introduce a two-stage adjustment scheme to refine the proposal boundaries and measure their confidence in containing an action with snippet-level actionness. Extensive experiments on the challenging ActivityNet and THUMOS14 benchmarks demonstrate our RapNet generates superior accurate proposals over the existing state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document