Dialogue Act Semantic Representation and Classification Using Recurrent Neural Networks

2017 ◽  
Author(s):  
Pinelopi Papalampidi ◽  
Elias Iosif ◽  
Alexandros Potamianos
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Semantic Representation

scholarly journals Joint Semantic Synthesis and Morphological Analysis of the Derived Word

2018 ◽  
Vol 6 ◽  
pp. 33-48 ◽  
Author(s):  
Ryan Cotterell ◽  
Hinrich Schütze
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Probabilistic Model ◽  
Morphological Analysis ◽  
Semantic Representation ◽  
English Word ◽  
Word Formation ◽  
Additive Models ◽  
Segmentation Accuracy ◽  
Morphological Productivity

Much like sentences are composed of words, words themselves are composed of smaller units. For example, the English word questionably can be analyzed as question+ able+ ly. However, this structural decomposition of the word does not directly give us a semantic representation of the word’s meaning. Since morphology obeys the principle of compositionality, the semantics of the word can be systematically derived from the meaning of its parts. In this work, we propose a novel probabilistic model of word formation that captures both the analysis of a word w into its constituent segments and the synthesis of the meaning of w from the meanings of those segments. Our model jointly learns to segment words into morphemes and compose distributional semantic vectors of those morphemes. We experiment with the model on English CELEX data and German DErivBase (Zeller et al., 2013) data. We show that jointly modeling semantics increases both segmentation accuracy and morpheme F1 by between 3% and 5%. Additionally, we investigate different models of vector composition, showing that recurrent neural networks yield an improvement over simple additive models. Finally, we study the degree to which the representations correspond to a linguist’s notion of morphological productivity.

Levenshtein Augmentation Improves Performance of SMILES Based Deep-Learning Synthesis Prediction

2020 ◽  
Author(s):  
Dean Sumner ◽  
Jiazhen He ◽  
Amol Thakkar ◽  
Ola Engkvist ◽  
Esben Jannik Bjerrum
Keyword(s):  
Neural Networks ◽  
Pattern Recognition ◽  
Deep Learning ◽  
Recurrent Neural Networks ◽  
Data Augmentation ◽  
State Of The Art ◽  
Sequence Similarity ◽  
Learning Models ◽  
Underlying Network

<p>SMILES randomization, a form of data augmentation, has previously been shown to increase the performance of deep learning models compared to non-augmented baselines. Here, we propose a novel data augmentation method we call “Levenshtein augmentation” which considers local SMILES sub-sequence similarity between reactants and their respective products when creating training pairs. The performance of Levenshtein augmentation was tested using two state of the art models - transformer and sequence-to-sequence based recurrent neural networks with attention. Levenshtein augmentation demonstrated an increase performance over non-augmented, and conventionally SMILES randomization augmented data when used for training of baseline models. Furthermore, Levenshtein augmentation seemingly results in what we define as <i>attentional gain </i>– an enhancement in the pattern recognition capabilities of the underlying network to molecular motifs.</p>

LatticeRnn: Recurrent Neural Networks Over Lattices

2016 ◽  
Author(s):  
Faisal Ladhak ◽  
Ankur Gandhe ◽  
Markus Dreyer ◽  
Lambert Mathias ◽  
Ariya Rastrow ◽  
...  
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks
Sign in / Sign up

Export Citation Format

Share Document