Forward-reverse expectation-maximization algorithm for Markov chains: convergence and numerical analysis

2018 ◽  
Vol 50 (2) ◽  
pp. 621-644
Author(s):  
Christian Bayer ◽  
Hilmar Mai ◽  
John Schoenmakers
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Expectation Maximization ◽  
Complexity Analysis ◽  
Markov Chain Model ◽  
Expectation Maximization Algorithm ◽  
Almost Sure Convergence ◽  
Terminal State ◽  
Chain Model ◽  
Curved Exponential Family

Abstract We develop a forward-reverse expectation-maximization (FREM) algorithm for estimating parameters of a discrete-time Markov chain evolving through a certain measurable state-space. For the construction of the FREM method, we develop forward-reverse representations for Markov chains conditioned on a certain terminal state. We prove almost sure convergence of our algorithm for a Markov chain model with curved exponential family structure. On the numerical side, we carry out a complexity analysis of the forward-reverse algorithm by deriving its expected cost. Two application examples are discussed.

scholarly journals Determination of the Order of a Markov Chain for Daily Rainfall Data of North East India: "Application AIC criterion"

2018 ◽  
Vol 10 (1) ◽  
pp. 80-87
Author(s):  
Surobhi Deka
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Akaike Information Criterion ◽  
Markov Chain Model ◽  
Information Criterion ◽  
Chain Model ◽  
Adequate Model ◽  
First Order ◽  
North East India ◽  
North East

The paper aims at demonstrating the application of the Akaike information criterion to determine the order of two state Markov chain for studying the pattern of occurrence of wet and dry days during the rainy season (April to September) in North-East India. For each station, each day is classified as dry day if the amount of rainfall is less than 3 mm and wet day if the amount of rainfall is greater than or equal to 3 mm. We apply Markov chain of order up to three to the sequences of wet and dry days observed at seven distantly located stations in North East region of India. The Markov chain model of appropriate order for analyzing wet and dry days is determined. This is done using the Akaike Information Criterion (AIC) by checking the minimum of AIC estimate. Markov chain of order one is found to be superior to the majority of the stations in comparison to the other order Markov chains. More precisely, first order Markov chain model is an adequate model for the stations North Bank, Tocklai, Silcoorie, Mohanbari and Guwahati. Further, it is observed that second order and third order Markov chains are competing with first order in the stations Cherrapunji and Imphal, respectively. A fore-knowledge of rainfall pattern is of immense help not only to farmers, but also to the authorities concerned with planning of irrigation schemes. The outcomes are useful for taking decisions well in advance for transplanting of rice as well as for other input management and farm activities during different stages of the crop growing season.

scholarly journals Higher order Markov Chain Model for Synthetic Generation of Daily Streamflows

2018 ◽  
Vol 19 (3) ◽  
pp. 449
Author(s):  
A. G. C. Pereira ◽  
F. A. S. Sousa ◽  
B. B. Andrade ◽  
Viviane Simioli Medeiros Campos
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Markov Chain Model ◽  
Chain Model ◽  
Order Markov Chain ◽  
Chain Order ◽  
Higher Order Markov Chain ◽  
Best Fit ◽  
Synthetic Generation ◽  
Better Than

The aim of this study is to get further into the two-state Markov chain model for synthetic generation daily streamflows. The model proposed in Aksoy and Bayazit (2000) and Aksoy (2003) is based on a two Markov chains for determining the state of the stream. The ascension curve of the hydrograph is modeled by a two-parameter Gamma probability distribution function and is assumed that a recession curve of the hydrograph follows an exponentially function. In this work, instead of assuming a pre-defined order for the Markov chains involved in the modelling of streamflows, a BIC test is performed to establish the Markov chain order that best fit on the data. The methodology was applied to data from seven Brazilian sites. The model proposed here was  better than that one proposed by Aksoy but for two sites which have the lowest time series and are located in the driest regions.

scholarly journals Nonlinear Markov Chain Modelling of the Novel Coronavirus (Covid-19) Pandemic

2020 ◽  
Author(s):  
Muammer Catak ◽  
Necati Duran
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Markov Chain Model ◽  
Chain Model ◽  
The Novel ◽  
The World ◽  
The Usa ◽  
Novel Coronavirus ◽  
Almost All ◽  
Markov Chain Modelling

Almost all countries around the world are struggling against the novel coronavirus (Covid-19) pandemic. In this paper, a nonlinear Markov chains model is proposed in order to analyse and to understand the behaviour of the Covid-19 pandemic. The data from China was used to build up the presented model. Thereafter, the nonlinear Markov chain model is employed to estimate the daily new Covid-19 cases in some countries including Italy, Spain, France, UK, the USA, Germany, Turkey, and Kuwait. In addition, the correlation between the daily new Covid-19 cases and the daily number of deaths is examined.

scholarly journals Expectation-Maximization Tensor Factorization for Practical Location Privacy Attacks

2017 ◽  
Vol 2017 (4) ◽  
pp. 138-155 ◽  
Author(s):  
Takao Murakami
Keyword(s):  
Markov Chain ◽  
Expectation Maximization ◽  
Location Privacy ◽  
Viterbi Algorithm ◽  
Markov Chain Model ◽  
Chain Model ◽  
Tensor Factorization ◽  
Learning Methods ◽  
Realistic Situation ◽  
Target User

Abstract Location privacy attacks based on a Markov chain model have been widely studied to de-anonymize or de-obfuscate mobility traces. An adversary can perform various kinds of location privacy attacks using a personalized transition matrix, which is trained for each target user. However, the amount of training data available to the adversary can be very small, since many users do not disclose much location information in their daily lives. In addition, many locations can be missing from the training traces, since many users do not disclose their locations continuously but rather sporadically. In this paper, we show that the Markov chain model can be a threat even in this realistic situation. Specifically, we focus on a training phase (i.e. mobility profile building phase) and propose Expectation-Maximization Tensor Factorization (EMTF), which alternates between computing a distribution of missing locations (E-step) and computing personalized transition matrices via tensor factorization (M-step). Since the time complexity of EMTF is exponential in the number of missing locations, we propose two approximate learning methods, one of which uses the Viterbi algorithm while the other uses the Forward Filtering Backward Sampling (FFBS) algorithm. We apply our learning methods to a de-anonymization attack and a localization attack, and evaluate them using three real datasets. The results show that our learning methods significantly outperform a random guess, even when there is only one training trace composed of 10 locations per user, and each location is missing with probability 80% (i.e. even when users hardly disclose two temporally-continuous locations).

A Comparison of Stochastic Models for the Interarrival Times of Packets in a Computer Network

2011 ◽  
pp. 248-257 ◽  
Author(s):  
Dennis Guster ◽  
Semyon Litvinov ◽  
Mary Richardson ◽  
David Robinson
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Power Law ◽  
Stochastic Models ◽  
Computer Network ◽  
Markov Chain Model ◽  
Actual Data ◽  
Chain Model ◽  
Power Law Process ◽  
Simulation Techniques

Because of the complexity and over-subscription of today’s networks, the importance of valid simulation techniques to aid in determining sound network design is paramount. A number of studies have shown that the theoretical exponential packet interarrival rates are not appropriate for many network installations. This chapter compares two other modeling techniques: the power law process and Markov chains to the exponential and actual data taken from a ten-minute segment. The results reveal that the exponential and power law models are a poor match to the actual data. The Markov chain model, although not perfect, yielded some promising results.

Application of Markov Chains to Identification of Turbine Engine Dynamic Models

2002 ◽  
Author(s):  
V. Yu. Arkov ◽  
G. G. Kulikov ◽  
T. V. Breikin
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Gas Turbines ◽  
Dynamic Models ◽  
Markov Chain Model ◽  
Dynamic Modelling ◽  
Parametric Models ◽  
Chain Model ◽  
Model Estimation ◽  
Identifiability Analysis

The paper addresses the problem of dynamic modelling of gas turbines for condition monitoring purposes. Identification of dynamic models is performed using a novel Markov chain technique. This includes identifiability analysis and model estimation. When identifying the model, experimental data should be sufficiently informative for identification. So far, identifiability analysis is weak formed and workable solutions are still to be developed. A possible technique is proposed based on non-parametric models in the form of controllable Markov chains. The second step in systems identification is the model estimation. At this stage, Markov chains are introduced to provide more functionality and versatility for dynamic modelling of gas turbines. The Markov chain model combines the deterministic and stochastic components of the engine dynamics within a single model, thus providing more exact and adequate description of the real system behaviour and leading to far more accurate health monitoring.

scholarly journals Semantic-Enhanced Multi-Dimensional Markov Chains on Semantic Trajectories for Predicting Future Locations

Sensors ◽  
2018 ◽  
Vol 18 (10) ◽  
pp. 3582 ◽  
Author(s):  
Antonios Karatzoglou ◽  
Dominik Köhler ◽  
Michael Beigl
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Markov Chain Model ◽  
Predictive Performance ◽  
Chain Model ◽  
Semantic Level ◽  
Trajectory Mining ◽  
Solid Foundation ◽  
Semantic Similarity Analysis ◽  
High Level

In this work, we investigate the performance of Markov Chains with respect to modelling semantic trajectories and predicting future locations. In the first part, we examine whether and to what degree the semantic level of semantic trajectories affects the predictive performance of a spatial Markov model. It can be shown that the choice of the semantic level when describing trajectories has a significant impact on the accuracy of the models. High-level descriptions lead to better results than low-level ones. The second part introduces a multi-dimensional Markov Chain construct that considers, besides locations, additional context information, such as time, day and the users’ activity. While the respective approach is able to outperform our baseline, we could also identify some limitations. These are mainly attributed to its sensitivity towards small-sized training datasets. We attempt to overcome this issue, among others, by adding a semantic similarity analysis component to our model that takes the varying role of locations due each time to the respective purpose of visiting the particular location explicitly into consideration. To capture the aforementioned dynamics, we define an entity, which we refer to as Purpose-of-Visit-Dependent Frame (PoVDF). In the third part of this work, we describe in detail the PoVDF-based approach and we evaluate it against the multi-dimensional Markov Chain model as well as with a semantic trajectory mining and prefix tree based model. Our evaluation shows that the PoVDF-based approach outperforms its competition and lays a solid foundation for further investigation.

scholarly journals Prediction of annual precipitation on the territory of south Serbia using Markov chains

2013 ◽  
pp. 81-92 ◽  
Author(s):  
Predrag Lukic ◽  
Milan Gocic ◽  
Slavisa Trajkovic
Keyword(s):  
Markov Chain ◽  
Markov Chains ◽  
Environmental Protection ◽  
Markov Chain Model ◽  
Stochastic Method ◽  
Annual Precipitation ◽  
Chain Model ◽  
Precipitation Data

Prediction of precipitation is one of the important factors that affect the sectors such as industry, agriculture, environmental protection, and their related fields. The stochastic method based on a Markov chain model is used in the paper to predict the annual precipitation in the territory of South Serbia for the period 2009-2013. For this purpose, the precipitation data rainfall recorded on the four synoptic stations were used for the period 1980-2010.

Sign in / Sign up

Export Citation Format

Share Document