Large vocabulary SOUL neural network language models

2011 ◽  
Author(s):  
Hai-Son Le ◽  
Ilya Oparin ◽  
Abdel Messaoudi ◽  
Alexandre Allauzen ◽  
Jean-Luc Gauvain ◽  
...  
Keyword(s):  
Neural Network ◽  
Language Models ◽  
Large Vocabulary ◽  
Network Language

scholarly journals Efficient Embedded Decoding of Neural Network Language Models in a Machine Translation System

2018 ◽  
Vol 28 (09) ◽  
pp. 1850007
Author(s):  
Francisco Zamora-Martinez ◽  
Maria Jose Castro-Bleda
Keyword(s):  
Neural Network ◽  
Machine Translation ◽  
Language Processing ◽  
Traditional Approach ◽  
Computational Cost ◽  
Integrated Approach ◽  
Language Models ◽  
Translation System ◽  
Neural Net ◽  
Network Language

Neural Network Language Models (NNLMs) are a successful approach to Natural Language Processing tasks, such as Machine Translation. We introduce in this work a Statistical Machine Translation (SMT) system which fully integrates NNLMs in the decoding stage, breaking the traditional approach based on [Formula: see text]-best list rescoring. The neural net models (both language models (LMs) and translation models) are fully coupled in the decoding stage, allowing to more strongly influence the translation quality. Computational issues were solved by using a novel idea based on memorization and smoothing of the softmax constants to avoid their computation, which introduces a trade-off between LM quality and computational cost. These ideas were studied in a machine translation task with different combinations of neural networks used both as translation models and as target LMs, comparing phrase-based and [Formula: see text]-gram-based systems, showing that the integrated approach seems more promising for [Formula: see text]-gram-based systems, even with nonfull-quality NNLMs.

scholarly journals Feature Based Domain Adaptation for Neural Network Language Models with Factorised Hidden Layers

2019 ◽  
Vol E102.D (3) ◽  
pp. 598-608 ◽  
Author(s):  
Michael HENTSCHEL ◽  
Marc DELCROIX ◽  
Atsunori OGAWA ◽  
Tomoharu IWATA ◽  
Tomohiro NAKATANI
Keyword(s):  
Neural Network ◽  
Domain Adaptation ◽  
Language Models ◽  
Feature Based ◽  
Network Language
Sign in / Sign up

Export Citation Format

Share Document