HMM-based Temporal Difference Learning with State Transition Updating for Tracking Human Communicational Behaviors

2003 ◽  
Vol 15 (3) ◽  
pp. 271-277 ◽  
Author(s):  
Minh Anh T. Ho ◽  
◽  
Yoji Yamada ◽  
Yoji Umetani
Keyword(s):  
State Transition ◽  
Markov Models ◽  
Transition Probability ◽  
Original System ◽  
Temporal Difference ◽  
Value Functions ◽  
Constraint Factor ◽  
Organization Framework ◽  
Sign Sequence ◽  
Updating Procedure

In our original system, we used hidden Markov models (HMMs) to model rough gesture patterns. We later utilized temporal difference (TD) learning to adjust the action model of the tracker for its behavior in the tracking task. We integrated the above two methods into an algorithm by assigning state transition probability in HMMs as a reward in TD learning. Identification of the sign gesture context through wavelet analysis autonomously provides a reward value for optimizing the attentive visual attentive tracker's AVAT's action patterns. A bound of state value functions as a constraint factor for the updating procedure in TD models has been determined to recognize whether predictive models need to be updated according with action models. Experimental results of extracting an operator's hand sign sequence during natural walking demonstrates AVAT development in the perceptual organization framework.

Asynchronous quadratic control for constrained hidden markov jump linear systems with incomplete MTPM and MOCPM

2021 ◽  
Author(s):  
Jin Zhu ◽  
Kai Xia ◽  
Geir E Dullerud
Keyword(s):  
Linear Systems ◽  
Controller Design ◽  
Hidden Markov ◽  
Transition Probability ◽  
Original System ◽  
Transition Probability Matrix ◽  
Markov Jump ◽  
Jump Linear Systems ◽  
Weighting Matrix ◽  
Markov Jump Linear Systems

Abstract This paper investigates the quadratic optimal control problem for constrained Markov jump linear systems with incomplete mode transition probability matrix (MTPM). Considering original system mode is not accessible, observed mode is utilized for asynchronous controller design where mode observation conditional probability matrix (MOCPM), which characterizes the emission between original modes and observed modes is assumed to be partially known. An LMI optimization problem is formulated for such constrained hidden Markov jump linear systems with incomplete MTPM and MOCPM. Based on this, a feasible state-feedback controller can be designed with the application of free-connection weighting matrix method. The desired controller, dependent on observed mode, is an asynchronous one which can minimize the upper bound of quadratic cost and satisfy restrictions on system states and control variables. Furthermore, clustering observation where observed modes recast into several clusters, is explored for simplifying the computational complexity. Numerical examples are provided to illustrate the validity.

scholarly journals Grey theory–based BP-NN co-training for dense sequence long-term tendency prediction

2020 ◽  
Vol ahead-of-print (ahead-of-print) ◽  
Author(s):  
Yuling Hong ◽  
Yingjie Yang ◽  
Qishan Zhang
Keyword(s):  
State Transition ◽  
Transition Probability ◽  
Training Model ◽  
Transition Probability Matrix ◽  
Content Type ◽  
Fine Grained ◽  
State Transition Probability ◽  
Dense Sequence ◽  
Popularity Prediction

PurposeThe purpose of this paper is to solve the problems existing in topic popularity prediction in online social networks and advance a fine-grained and long-term prediction model for lack of sufficient data.Design/methodology/approachBased on GM(1,1) and neural networks, a co-training model for topic tendency prediction is proposed in this paper. The interpolation based on GM(1,1) is employed to generate fine-grained prediction values of topic popularity time series and two neural network models are considered to achieve convergence by transmitting training parameters via their loss functions.FindingsThe experiment results indicate that the integrated model can effectively predict dense sequence with higher performance than other algorithms, such as NN and RBF_LSSVM. Furthermore, the Markov chain state transition probability matrix model is used to improve the prediction results.Practical implicationsFine-grained and long-term topic popularity prediction, further improvement could be made by predicting any interpolation in the time interval of popularity data points.Originality/valueThe paper succeeds in constructing a co-training model with GM(1,1) and neural networks. Markov chain state transition probability matrix is deployed for further improvement of popularity tendency prediction.

Extracting State Transition Dynamics from Multiple Spike Trains Using Hidden Markov Models with Correlated Poisson Distribution

2010 ◽  
Vol 22 (9) ◽  
pp. 2369-2389 ◽  
Author(s):  
Kentaro Katahira ◽  
Jun Nishikawa ◽  
Kazuo Okanoya ◽  
Masato Okada
Keyword(s):  
Hidden Markov Models ◽  
Poisson Distribution ◽  
State Transition ◽  
Markov Models ◽  
Hidden Markov ◽  
Synthetic Data ◽  
Poisson Model ◽  
Spike Trains ◽  
Output Distribution ◽  
Multivariate Poisson Distribution

Neural activity is nonstationary and varies across time. Hidden Markov models (HMMs) have been used to track the state transition among quasi-stationary discrete neural states. Within this context, an independent Poisson model has been used for the output distribution of HMMs; hence, the model is incapable of tracking the change in correlation without modulating the firing rate. To achieve this, we applied a multivariate Poisson distribution with correlation terms for the output distribution of HMMs. We formulated a variational Bayes (VB) inference for the model. The VB could automatically determine the appropriate number of hidden states and correlation types while avoiding the overlearning problem. We developed an efficient algorithm for computing posteriors using the recursive relationship of a multivariate Poisson distribution. We demonstrated the performance of our method on synthetic data and real spike trains recorded from a songbird.

Sign in / Sign up

Export Citation Format

Share Document