Markov Decision Models for Optimal Decision Making in Bottlenecks Identification and Mitigation

Existing methods for bottleneck detection can be categorized into two: methods based on stochastic analysis and methods based on data-driven analysis. The stochastic methods are accurate in estimating bottlenecks in long term, ignoring the current improvement opportunities, while the data-driven methods tend to do the opposite. In this paper, we develop an optimal policy to integrate the two methods based on Markov decision theory. The characterization of the optimal policy is provided. In addition, to implement the policy, the optimal frequency for carrying out bottleneck analysis is investigated. Numerical experiment is performed to validate the effectiveness of the optimal policy and compare it to the existing methods.

IDENTIFICATION IN DISCRETE MARKOV DECISION MODELS

We derive conditions for the identification of the structural parameters in Markov decision model under the assumptions of Rust (1987, Econometrica 55, 999–1033) when the payoff function is parametrically specified. Identification in this class of dynamic problems is difficult to establish since the parameters of interest enter the value function nonlinearly, and the value function is only defined implicitly as a fixed point of some functional equation. We show it is sufficient to verify identification in the pseudomodel, which is more tractable as it is originally designed to reduce the computational burden in the estimation problem, for the identification of the data generating parameter of the underling model. Our results extend naturally to a class of dynamic discrete action games commonly used in empirical industrial organizations.