scholarly journals Dialog History Construction with Long-Short Term Memory for Robust Generative Dialog State Tracking

2016 ◽  
Vol 7 (3) ◽  
pp. 47-64 ◽  
Author(s):  
Byung-Jun Lee ◽  
Kee-Eung Kim
Keyword(s):  
Speech Processing ◽  
Short Term Memory ◽  
Dialog Systems ◽  
Dialog System ◽  
Gradient Descent Algorithm ◽  
Core Areas ◽  
Overall Performance ◽  
Long Short Term Memory ◽  
State Tracking ◽  
Dialog State Tracking

One of the crucial components of dialog system is the dialog state tracker, which infers user’s intention from preliminary speech processing. Since the overall performance of the dialog system is heavily affected by that of the dialog tracker, it has been one of the core areas of research on dialog systems. In this paper, we present a dialog state tracker that combines a generative probabilistic model of dialog state tracking with the recurrent neural network for encoding important aspects of the dialog history. We describe a two-step gradient descent algorithm that optimizes the tracker with a complex loss function. We demonstrate that this approach yields a dialog state tracker that performs competitively with top-performing trackers participated in the first and second Dialog State Tracking Challenges.

scholarly journals A Two-Step Neural Dialog State Tracker for Task-Oriented Dialog Processing

2018 ◽  
Vol 2018 ◽  
pp. 1-11
Author(s):  
A-Yeong Kim ◽  
Hyun-Je Song ◽  
Seong-Bae Park
Keyword(s):  
Attention Mechanism ◽  
Data Set ◽  
Dialog Systems ◽  
Dialog System ◽  
Fast Training ◽  
Proposed Model ◽  
Spoken Dialog System ◽  
State Tracking ◽  
Dialog State Tracking ◽  
Task Oriented

Dialog state tracking in a spoken dialog system is the task that tracks the flow of a dialog and identifies accurately what a user wants from the utterance. Since the success of a dialog is influenced by the ability of the system to catch the requirements of the user, accurate state tracking is important for spoken dialog systems. This paper proposes a two-step neural dialog state tracker which is composed of an informativeness classifier and a neural tracker. The informativeness classifier which is implemented by a CNN first filters out noninformative utterances in a dialog. Then, the neural tracker estimates dialog states from the remaining informative utterances. The tracker adopts the attention mechanism and the hierarchical softmax for its performance and fast training. To prove the effectiveness of the proposed model, we do experiments on dialog state tracking in the human-human task-oriented dialogs with the standard DSTC4 data set. Our experimental results prove the effectiveness of the proposed model by showing that the proposed model outperforms the neural trackers without the informativeness classifier, the attention mechanism, or the hierarchical softmax.

scholarly journals The Dialog State Tracking Challenge Series

AI Magazine ◽  
2014 ◽  
Vol 35 (4) ◽  
pp. 121-124 ◽  
Author(s):  
Jason D. Williams ◽  
Matthew Henderson ◽  
Antoine Raux ◽  
Blaise Thomson ◽  
Alan Black ◽  
...  
Keyword(s):  
Research Community ◽  
Spoken Dialog Systems ◽  
Dialog Systems ◽  
New Methods ◽  
State Tracking ◽  
Dialog State Tracking

In spoken dialog systems, dialog state tracking refers to the task of correctly inferring the user's goal at a given turn, given all of the dialog history up to that turn. The Dialog State Tracking Challenge is a research community challenge task that has run for three rounds. The challenge has given rise to a host of new methods for dialog state tracking, and also deeper understandings about the problem itself, including methods for evaluation.

scholarly journals Spectral decomposition method of dialog state tracking via collective matrix factorization

2016 ◽  
Vol 7 (3) ◽  
pp. 34-46
Author(s):  
Julien Perez
Keyword(s):  
Matrix Factorization ◽  
The State ◽  
Computationally Efficient ◽  
Reward Function ◽  
Dialog Management ◽  
Dialog System ◽  
Dependent Variables ◽  
Novel Method ◽  
State Tracking ◽  
Dialog State Tracking

The task of dialog management is commonly decomposed into two sequential subtasks: dialog state tracking and dialog policy learning. In an end-to-end dialog system, the aim of dialog state tracking is to accurately estimate the true dialog state from noisy observations produced by the speech recognition and the natural language understanding modules. The state tracking task is primarily meant to support a dialog policy. From a probabilistic perspective, this is achieved by maintaining a posterior distribution over hidden dialog states composed of a set of context dependent variables. Once a dialog policy is learned, it strives to select an optimal dialog act given the estimated dialog state and a defined reward function. This paper introduces a novel method of dialog state tracking based on a bilinear algebric decomposition model that provides an efficient inference schema through collective matrix factorization. We evaluate the proposed approach on the second Dialog State Tracking Challenge (DSTC-2) dataset and we show that the proposed tracker gives encouraging results compared to the state-of-the-art trackers that participated in this standard benchmark. Finally, we show that the prediction schema is computationally efficient in comparison to the previous approaches.

scholarly journals A Meta-Modeling Power Consumption Forecasting Approach Combining Client Similarity and Causality

Energies ◽  
2021 ◽  
Vol 14 (19) ◽  
pp. 6088
Author(s):  
Dimitrios Kontogiannis ◽  
Dimitrios Bargiotas ◽  
Aspassia Daskalopulu ◽  
Lefteri H. Tsoukalas
Keyword(s):  
Power Consumption ◽  
Short Term Memory ◽  
Electricity Consumption ◽  
Percentage Error ◽  
Forecasting Models ◽  
Novel Approach ◽  
Data Collection Process ◽  
Meta Modeling ◽  
Overall Performance ◽  
Long Short Term Memory

Power forecasting models offer valuable insights on the electricity consumption patterns of clients, enabling the development of advanced strategies and applications aimed at energy saving, increased energy efficiency, and smart energy pricing. The data collection process for client consumption models is not always ideal and the resulting datasets often lead to compromises in the implementation of forecasting models, as well as suboptimal performance, due to several challenges. Therefore, combinations of elements that highlight relationships between clients need to be investigated in order to achieve more accurate consumption predictions. In this study, we exploited the combined effects of client similarity and causality, and developed a power consumption forecasting model that utilizes ensembles of long short-term memory (LSTM) networks. Our novel approach enables the derivation of different representations of the predicted consumption based on feature sets influenced by similarity and causality metrics. The resulting representations were used to train a meta-model, based on a multi-layer perceptron (MLP), in order to combine the results of the LSTM ensembles optimally. This combinatorial approach achieved better overall performance and yielded lower mean absolute percentage error when compared to the standalone LSTM ensembles that do not include similarity and causality. Additional experiments indicated that the combination of similarity and causality resulted in more performant models when compared to implementations utilizing only one element on the same model structure.

scholarly journals The Dialog State Tracking Challenge Series: A Review

2016 ◽  
Vol 7 (3) ◽  
pp. 4-33 ◽  
Author(s):  
Jason D. Williams ◽  
Antoine Raux ◽  
Matthew Henderson
Keyword(s):  
Speech Recognition ◽  
Research Area ◽  
The State ◽  
Evaluation Metrics ◽  
Common Resources ◽  
Discriminative Models ◽  
Dialog System ◽  
Spoken Dialog System ◽  
State Tracking ◽  
Dialog State Tracking

In a spoken dialog system, dialog state tracking refers to the task of correctly inferring the state of the conversation -- such as the user's goal -- given all of the dialog history up to that turn.  Dialog state tracking is crucial to the success of a dialog system, yet until recently there were no common resources, hampering progress.  The Dialog State Tracking Challenge series of 3 tasks introduced the first shared testbed and evaluation metrics for dialog state tracking, and has underpinned three key advances in dialog state tracking: the move from generative to discriminative models; the adoption of discriminative sequential techniques; and the incorporation of the speech recognition results directly into the dialog state tracker.  This paper reviews this research area, covering both the challenge tasks themselves and summarizing the work they have enabled.

Using Machine Learning Algorithms on Prediction of Stock Price

2020 ◽  
Vol 12 (2) ◽  
pp. 84-99
Author(s):  
Li-Pang Chen
Keyword(s):  
Machine Learning ◽  
Stock Price ◽  
Short Term Memory ◽  
Machine Learning Algorithms ◽  
Machine Learning Techniques ◽  
Support Vector ◽  
Short Term ◽  
Learning Techniques ◽  
Historical Database ◽  
Long Short Term Memory

In this paper, we investigate analysis and prediction of the time-dependent data. We focus our attention on four different stocks are selected from Yahoo Finance historical database. To build up models and predict the future stock price, we consider three different machine learning techniques including Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN) and Support Vector Regression (SVR). By treating close price, open price, daily low, daily high, adjusted close price, and volume of trades as predictors in machine learning methods, it can be shown that the prediction accuracy is improved.

Deep Reinforcement Learning for Multiparameter Optimization in de novo Drug Design

2019 ◽  
Author(s):  
Niclas Ståhl ◽  
Göran Falkman ◽  
Alexander Karlsson ◽  
Gunnar Mathiason ◽  
Jonas Boström
Keyword(s):  
Reinforcement Learning ◽  
Short Term Memory ◽  
De Novo ◽  
De Novo Drug Design ◽  
Generative Process ◽  
New Methods ◽  
Multiparameter Optimization ◽  
Long Short Term Memory ◽  
New Compounds

<p>In medicinal chemistry programs it is key to design and make compounds that are efficacious and safe. This is a long, complex and difficult multi-parameter optimization process, often including several properties with orthogonal trends. New methods for the automated design of compounds against profiles of multiple properties are thus of great value. Here we present a fragment-based reinforcement learning approach based on an actor-critic model, for the generation of novel molecules with optimal properties. The actor and the critic are both modelled with bidirectional long short-term memory (LSTM) networks. The AI method learns how to generate new compounds with desired properties by starting from an initial set of lead molecules and then improve these by replacing some of their fragments. A balanced binary tree based on the similarity of fragments is used in the generative process to bias the output towards structurally similar molecules. The method is demonstrated by a case study showing that 93% of the generated molecules are chemically valid, and a third satisfy the targeted objectives, while there were none in the initial set.</p>

Sign in / Sign up

Export Citation Format

Share Document