Using Hidden Markov Models to Determine Changes in Subject Data over Time, Studying the Immunoregulatory Effect of Mesenchymal Stem Cells

Reconciliation of inconsistent data sources using hidden Markov models

Statistical Journal of the IAOS ◽

10.3233/sji-190594 ◽

2020 ◽

Vol 36 (4) ◽

pp. 1261-1279

Author(s):

Paulina Pankowska ◽

Dimitris Pavlopoulos ◽

Bart Bakker ◽

Daniel L. Oberski

Keyword(s):

Measurement Error ◽

Data Collection ◽

Hidden Markov Models ◽

Markov Models ◽

Hidden Markov ◽

Parameter Estimates ◽

Collection Process ◽

Data Collection Process ◽

Linkage Error ◽

Over Time

This paper discusses how National Statistical Institutes (NSI’s) can use hidden Markov models (HMMs) to produce consistent official statistics for categorical, longitudinal variables using inconsistent sources. Two main challenges are addressed: first, the reconciliation of inconsistent sources with multi-indicator HMMs requires linking the sources on the micro level. Such linkage might lead to bias due to linkage error. Second, applying and estimating HMMs regularly is a complicated and expensive procedure. Therefore, it is preferable to use the error parameter estimates as a correction factor for a number of years. However, this might lead to biased structural estimates if measurement error changes over time or if the data collection process changes. Our results on these issues are highly encouraging and imply that the suggested method is appropriate for NSI’s. Specifically, linkage error only leads to (substantial) bias in very extreme scenarios. Moreover, measurement error parameters are largely stable over time if no major changes in the data collection process occur. However, when a substantial change in the data collection process occurs, such as a switch from dependent (DI) to independent (INDI) interviewing, re-using measurement error estimates is not advisable.

Download Full-text

Time stability of the impact of institutions on economic growth and real convergence of the EU countries: implications from the hidden Markov models analysis

Equilibrium ◽

10.24136/eq.2021.011 ◽

2021 ◽

Vol 16 (2) ◽

pp. 285-323

Author(s):

Michał Bernardelli ◽

Mariusz Próchniak ◽

Bartosz Witkowski

Keyword(s):

Economic Growth ◽

Hidden Markov Models ◽

Structural Breaks ◽

Markov Models ◽

Hidden Markov ◽

Value Added ◽

Time Stability ◽

The Impact ◽

Over Time ◽

The Eu

Research background: It is not straightforward to identify the role of institutions for the economic growth. The possible unknown or uncertain areas refer to nonlinearities, time stability, transmission channels, and institutional complementarities. The research problem tackled in this paper is the analysis of the time stability of the relationship between institutions and economic growth and real economic convergence. Purpose of the article: The article aims to verify whether the impact of the institutional environment on GDP dynamics was stable over time or diffed in various subperiods. The analysis covers the EU28 countries and the 1995?2019 period. Methods: We use regression equations with time dummies and interactions to assess the stability of the impact of institutions on economic growth. The analysis is based on the partially overlapping observations. The models are estimated with the use of Blundell and Bond?s GMM system estimator. The results are then averaged with the Bayesian Model Averaging (BMA) approach. Structural breaks are identified on the basis of the Hidden Markov Models (HMM). Findings & value added: The value added of the study is threefold. First, we use the HMM approach to find structural breaks. Second, the BMA method is applied to assess the robustness of the outcomes. Third, we show the potential of HMM in foresighting. The results of regression estimates indicate that good institution reflected in the greater scope of economic freedom and better governance lead to the higher economic growth of the EU countries. However, the impact of institutions on economic growth was not stable over time.

Download Full-text

Hidden Markov models and their applications in finance

10.12681/eadd/36701 ◽

2015 ◽

Author(s):

Αναστάσιος Πετρόπουλος

Keyword(s):

Hidden Markov Models ◽

Financial Institutions ◽

Markov Models ◽

Temporal Dynamics ◽

Hidden Markov ◽

Credit Scoring ◽

Change Over Time ◽

Corporate Credit ◽

Over Time ◽

Temporal Dependencies

Hidden Markov Models, usually referred to as HMMs, are one of the most successful concepts in statistical modeling conceived and analyzed in the last 40 years. They belong to the stochastic mixture models family and have been broadly implemented in numerous sectors to address the problem of data model fitting and forecasting. Their structure usually is comprised by an observed sequence which is conditioned on an underlying hidden (unobserved) process. This way HMMs provide flexibility to address various complicated problems and can be implemented for modeling univariate and multivariate financial time series. Moreover, based on current literature, economic variables exhibit patterns dependent on different economic regimes which can be successfully captured by HMMs. Their parsimonious structure and attractive properties along with the existence of efficient algorithms for their estimation were the main drivers for the selection of HMM as the main topic of this thesis. Consequently, in this thesis we thoroughly investigate HMMs and their capabilities to simulate financial systems. The contribution of this study is threefold: First we perform an extensive review of HMM theory and applications. Our aim is to summarize the most significant applications of HMM with special focus in the field of finance. We offer a thorough and compact summary of the uses and the results of HMM in the last 40 years. Secondly, we extend the framework of HMMs by proposing a theoretical variation, injecting greater flexibility in their structure. Based on bibliography, in many real-world scenarios the modeled data entail temporal dynamics the patterns of which change over time. We address this problem by proposing a novel HMM formulation, treating temporal dependencies as latent variables over which inference is performed. Specifically, we introduce a hierarchical graphical model comprising two hidden layers: on the first layer, we postulate a chain of latent observation-emitting states, the temporal dependencies between which may change over time; on the second layer, we postulate a latent first-order Markov chain modeling the evolution of temporal dynamics (dependence jumps) pertaining to the first-layer latent process. As a result of this construction, our method allows for effectively modeling non-homogeneous observed financial data. Finally in the third part of this thesis we investigate the HMM efficiency in the problem of corporate credit scoring. We propose a novel corporate credit rating system based on Student’s-t hidden Markov models (SHMMs). Corporate credit scoring is widely used by financial institutions for portfolio risk management, and for pricing financial products designed for corporations. In addition, from a regulatory perspective, internal rating models are commonly used for establishing a more risk-sensitive capital adequacy framework for financial institutions. We evaluate our method against other state of the art statistical techniques like Neural Networks, SVM, and logistic regression and conclude that SHMM offer significant improved forecasting capabilities.

Download Full-text

Estimating Personality Impression from Speech Record Using Hidden Markov Models

IEEJ Transactions on Electronics Information and Systems ◽

10.1541/ieejeiss.135.1517 ◽

2015 ◽

Vol 135 (12) ◽

pp. 1517-1523 ◽

Cited By ~ 1

Author(s):

Yicheng Jin ◽

Takuto Sakuma ◽

Shohei Kato ◽

Tsutomu Kunitachi

Keyword(s):

Hidden Markov Models ◽

Markov Models ◽

Hidden Markov

Download Full-text

Hidden Markov Processes

10.23943/princeton/9780691133157.001.0001 ◽

2014 ◽

Cited By ~ 2

Author(s):

M. Vidyasagar

Keyword(s):

Hidden Markov Models ◽

Markov Processes ◽

Viterbi Algorithm ◽

Markov Models ◽

Hidden Markov ◽

Local Alignment ◽

Biological Applications ◽

Standard Material ◽

Hidden Markov Processes ◽

Genomics And Proteomics

This book explores important aspects of Markov and hidden Markov processes and the applications of these ideas to various problems in computational biology. It starts from first principles, so that no previous knowledge of probability is necessary. However, the work is rigorous and mathematical, making it useful to engineers and mathematicians, even those not interested in biological applications. A range of exercises is provided, including drills to familiarize the reader with concepts and more advanced problems that require deep thinking about the theory. Biological applications are taken from post-genomic biology, especially genomics and proteomics. The topics examined include standard material such as the Perron–Frobenius theorem, transient and recurrent states, hitting probabilities and hitting times, maximum likelihood estimation, the Viterbi algorithm, and the Baum–Welch algorithm. The book contains discussions of extremely useful topics not usually seen at the basic level, such as ergodicity of Markov processes, Markov Chain Monte Carlo (MCMC), information theory, and large deviation theory for both i.i.d and Markov processes. It also presents state-of-the-art realization theory for hidden Markov models. Among biological applications, it offers an in-depth look at the BLAST (Basic Local Alignment Search Technique) algorithm, including a comprehensive explanation of the underlying theory. Other applications such as profile hidden Markov models are also explored.

Download Full-text