scholarly journals DanQ: a hybrid convolutional and recurrent deep neural network for quantifying the function of DNA sequences

2015 ◽  
Author(s):  
Daniel Quang ◽  
Xiaohui Xie
Keyword(s):  
Neural Network ◽  
Dna Sequences ◽  
Short Term Memory ◽  
De Novo ◽  
Short Term ◽  
Noncoding Dna ◽  
Regulatory Motifs ◽  
Precision Recall Curve ◽  
Long Short Term Memory

Modeling the properties and functions of DNA sequences is an important, but challenging task in the broad field of genomics. This task is particularly difficult for noncoding DNA, the vast majority of which is still poorly understood in terms of function. A powerful predictive model for the function of noncoding DNA can have enormous benefit for both basic science and translational research because over 98% of the human genome is noncoding and 93% of disease-associated variants lie in these regions. To address this need, we propose DanQ, a novel hybrid convolutional and bi-directional long short-term memory recurrent neural network framework for predicting noncoding function de novo from sequence. In the DanQ model, the convolution layer captures regulatory motifs, while the recurrent layer captures long-term dependencies between the motifs in order to learn a regulatory "grammar" to improve predictions. DanQ improves considerably upon other models across several metrics. For some regulatory markers, DanQ can achieve over a 50% relative improvement in the area under the precision-recall curve metric compared to related models.

scholarly journals Long Short Term Memory Recurrent Network for Standard and Poor’s 500 Index Modelling

2018 ◽  
Vol 7 (4.15) ◽  
pp. 25 ◽  
Author(s):  
Said Jadid Abdulkadir ◽  
Hitham Alhussian ◽  
Muhammad Nazmi ◽  
Asim A Elsheikh
Keyword(s):  
Neural Network ◽  
Time Series ◽  
Recurrent Neural Network ◽  
Time Series Data ◽  
Short Term Memory ◽  
Series Data ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Forecasting time-series data are imperative especially when planning is required through modelling using uncertain knowledge of future events. Recurrent neural network models have been applied in the industry and outperform standard artificial neural networks in forecasting, but fail in long term time-series forecasting due to the vanishing gradient problem. This study offers a robust solution that can be implemented for long-term forecasting using a special architecture of recurrent neural network known as Long Short Term Memory (LSTM) model to overcome the vanishing gradient problem. LSTM is specially designed to avoid the long-term dependency problem as their default behavior. Empirical analysis is performed using quantitative forecasting metrics and comparative model performance on the forecasted outputs. An evaluation analysis is performed to validate that the LSTM model provides better forecasted outputs on Standard & Poor’s 500 Index (S&P 500) in terms of error metrics as compared to other forecasting models.  

scholarly journals Deep Learning for Subtyping and Prediction of Diseases: Long-Short Term Memory

2021 ◽  
Author(s):  
Hayrettin Okut
Keyword(s):  
Neural Network ◽  
Short Term Memory ◽  
Short Term ◽  
Time Step ◽  
Term Memory ◽  
The Neural Network ◽  
Long Short Term Memory ◽  
Sequential Information ◽  
Hidden Layer

The long short-term memory neural network (LSTM) is a type of recurrent neural network (RNN). During the training of RNN architecture, sequential information is used and travels through the neural network from input vector to the output neurons, while the error is calculated and propagated back through the network to update the network parameters. Information in these networks incorporates loops into the hidden layer. Loops allow information to flow multi-directionally so that the hidden state signifies past information held at a given time step. Consequently, the output is dependent on the previous predictions which are already known. However, RNNs have limited capacity to bridge more than a certain number of steps. Mainly this is due to the vanishing of gradients which causes the predictions to capture the short-term dependencies as information from earlier steps decays. As more layers in RNN containing activation functions are added, the gradient of the loss function approaches zero. The LSTM neural networks (LSTM-ANNs) enable learning long-term dependencies. LSTM introduces a memory unit and gate mechanism to enable capture of the long dependencies in a sequence. Therefore, LSTM networks can selectively remember or forget information and are capable of learn thousands timesteps by structures called cell states and three gates.

scholarly journals Improving the Learning Power of Artificial Intelligence Using Multimodal Deep Learning

2021 ◽  
Vol 248 ◽  
pp. 01017
Author(s):  
Eugene Yu. Shchetinin ◽  
Leonid Sevastianov
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Short Term Memory ◽  
Short Term ◽  
Security Systems ◽  
Term Memory ◽  
Human Voice ◽  
Long Short Term Memory ◽  
Effective Analysis

Computer paralinguistic analysis is widely used in security systems, biometric research, call centers and banks. Paralinguistic models estimate different physical properties of voice, such as pitch, intensity, formants and harmonics to classify emotions. The main goal is to find such features that would be robust to outliers and will retain variety of human voice properties at the same time. Moreover, the model used must be able to estimate features on a time scale for an effective analysis of voice variability. In this paper a paralinguistic model based on Bidirectional Long Short-Term Memory (BLSTM) neural network is described, which was trained for vocal-based emotion recognition. The main advantage of this network architecture is that each module of the network consists of several interconnected layers, providing the ability to recognize flexible long-term dependencies in data, which is important in context of vocal analysis. We explain the architecture of a bidirectional neural network model, its main advantages over regular neural networks and compare experimental results of BLSTM network with other models.

scholarly journals RamaNet: Computational de novo helical protein backbone design using a long short-term memory generative neural network

F1000Research ◽  
2020 ◽  
Vol 9 ◽  
pp. 298 ◽  
Author(s):  
Sari Sabban ◽  
Mikhail Markovsky
Keyword(s):  
Neural Network ◽  
Protein Design ◽  
Short Term Memory ◽  
De Novo ◽  
Protein Structures ◽  
Protein Backbone ◽  
Short Term ◽  
Term Memory ◽  
Helical Protein ◽  
Long Short Term Memory

The ability to perform de novo protein design will allow researchers to expand the variety of available proteins. By designing synthetic structures computationally, they can utilise more structures than those available in the Protein Data Bank, design structures that are not found in nature, or direct the design of proteins to acquire a specific desired structure. While some researchers attempt to design proteins from first physical and thermodynamic principals, we decided to attempt to test whether it is possible to perform de novo helical protein design of just the backbone statistically using machine learning by building a model that uses a long short-term memory (LSTM) architecture. The LSTM model used only the φ and ψ angles of each residue from an augmented dataset of only helical protein structures. Though the network’s generated backbone structures were not perfect, they were idealised and evaluated post generation where the non-ideal structures were filtered out and the adequate structures kept. The results were successful in developing a logical, rigid, compact, helical protein backbone topology. This paper is a proof of concept that shows it is possible to generate a novel helical backbone topology using an LSTM neural network architecture using only the φ and ψ angles as features. The next step is to attempt to use these backbone topologies and sequence design them to form complete protein structures.

scholarly journals RamaNet: Computational de novo helical protein backbone design using a long short-term memory generative neural network

F1000Research ◽  
2020 ◽  
Vol 9 ◽  
pp. 298 ◽  
Author(s):  
Sari Sabban ◽  
Mikhail Markovsky
Keyword(s):  
Neural Network ◽  
Protein Design ◽  
Short Term Memory ◽  
De Novo ◽  
Protein Structures ◽  
Protein Backbone ◽  
Short Term ◽  
Term Memory ◽  
Helical Protein ◽  
Long Short Term Memory

The ability to perform de novo protein design will allow researchers to expand the variety of available proteins. By designing synthetic structures computationally, they can utilise more structures than those available in the Protein Data Bank, design structures that are not found in nature, or direct the design of proteins to acquire a specific desired structure. While some researchers attempt to design proteins from first physical and thermodynamic principals, we decided to attempt to test whether it is possible to perform de novo helical protein design of just the backbone statistically using machine learning by building a model that uses a long short-term memory (LSTM) architecture. The LSTM model used only the φ and ψ angles of each residue from an augmented dataset of only helical protein structures. Though the network’s generated backbone structures were not perfect, they were idealised and evaluated post generation where the non-ideal structures were filtered out and the adequate structures kept. The results were successful in developing a logical, rigid, compact, helical protein backbone topology. This paper is a proof of concept that shows it is possible to generate a novel helical backbone topology using an LSTM neural network architecture using only the φ and ψ angles as features. The next step is to attempt to use these backbone topologies and sequence design them to form complete protein structures.

SACNN: Self-attentive Convolutional Neural Network Model for Natural Language Inference

2021 ◽  
Vol 20 (3) ◽  
pp. 1-16
Author(s):  
Waris Quamer ◽  
Praphula Kumar Jain ◽  
Arpit Rai ◽  
Vijayalakshmi Saravanan ◽  
Rajendra Pamula ◽  
...  
Keyword(s):  
Neural Network ◽  
Natural Language ◽  
Short Term Memory ◽  
Local Context ◽  
Short Term ◽  
Proposed Model ◽  
Long Short Term Memory ◽  
Complex Relationships ◽  
Gated Recurrent Units

Inference has been central problem for understanding and reasoning in artificial intelligence. Especially, Natural Language Inference is an interesting problem that has attracted the attention of many researchers. Natural language inference intends to predict whether a hypothesis sentence can be inferred from the premise sentence. Most prior works rely on a simplistic association between the premise and hypothesis sentence pairs, which is not sufficient for learning complex relationships between them. The strategy also fails to exploit local context information fully. Long Short Term Memory (LSTM) or gated recurrent units networks (GRU) are not effective in modeling long-term dependencies, and their schemes are far more complex as compared to Convolutional Neural Networks (CNN). To address this problem of long-term dependency, and to involve context for modeling better representation of a sentence, in this article, a general Self-Attentive Convolution Neural Network (SACNN) is presented for natural language inference and sentence pair modeling tasks. The proposed model uses CNNs to integrate mutual interactions between sentences, and each sentence with their counterparts is taken into consideration for the formulation of their representation. Moreover, the self-attention mechanism helps fully exploit the context semantics and long-term dependencies within a sentence. Experimental results proved that SACNN was able to outperform strong baselines and achieved an accuracy of 89.7% on the stanford natural language inference (SNLI) dataset.

Sign in / Sign up

Export Citation Format

Share Document