Verification of General Markov Decision Processes by Approximate Similarity Relations and Policy Refinement

Sofie Haesaert; Sadegh Esmaeil Zadeh Soudjani; Alessandro Abate

doi:10.1137/16m1079397

Verification of General Markov Decision Processes by Approximate Similarity Relations and Policy Refinement

SIAM Journal on Control and Optimization ◽

10.1137/16m1079397 ◽

2017 ◽

Vol 55 (4) ◽

pp. 2333-2367 ◽

Cited By ~ 14

Author(s):

Sofie Haesaert ◽

Sadegh Esmaeil Zadeh Soudjani ◽

Alessandro Abate

Keyword(s):

Markov Decision Processes ◽

Decision Processes ◽

Similarity Relations ◽

Markov Decision ◽

Approximate Similarity ◽

Policy Refinement

Download Full-text

Verification of General Markov Decision Processes by Approximate Similarity Relations and Policy Refinement

Quantitative Evaluation of Systems - Lecture Notes in Computer Science ◽

10.1007/978-3-319-43425-4_16 ◽

2016 ◽

pp. 227-243 ◽

Cited By ~ 2

Author(s):

Sofie Haesaert ◽

Alessandro Abate ◽

Paul M. J. Van den Hof

Keyword(s):

Markov Decision Processes ◽

Decision Processes ◽

Similarity Relations ◽

Markov Decision ◽

Approximate Similarity ◽

Policy Refinement

Download Full-text

Temporal logic control of general Markov decision processes by approximate policy refinement

IFAC-PapersOnLine ◽

10.1016/j.ifacol.2018.08.013 ◽

2018 ◽

Vol 51 (16) ◽

pp. 73-78 ◽

Cited By ~ 4

Author(s):

Sofie Haesaert ◽

Sadegh Soudjani ◽

Alessandro Abate

Keyword(s):

Temporal Logic ◽

Markov Decision Processes ◽

Decision Processes ◽

Logic Control ◽

Markov Decision ◽

Policy Refinement

Download Full-text

Learning Control of Dynamical Systems Based on Markov Decision Processes: Research Frontiers and Outlooks

ACTA AUTOMATICA SINICA ◽

10.3724/sp.j.1004.2012.00673 ◽

2012 ◽

Vol 38 (5) ◽

pp. 673-687 ◽

Cited By ~ 1

Author(s):

Xin XU ◽

Dong SHEN ◽

Yan-Qing GAO ◽

Kai WANG

Keyword(s):

Dynamical Systems ◽

Markov Decision Processes ◽

Learning Control ◽

Decision Processes ◽

Markov Decision ◽

Research Frontiers

Download Full-text

A Framework for Modeling Bounded Rationality: Mis-Specified Bayesian-Markov Decision Processes

SSRN Electronic Journal ◽

10.2139/ssrn.2710475 ◽

2016 ◽

Cited By ~ 1

Author(s):

Ignacio Esponda ◽

Demian Pouzo

Keyword(s):

Bounded Rationality ◽

Markov Decision Processes ◽

Decision Processes ◽

Markov Decision

Download Full-text

A Vector Minimum Superharmonic Approach to Solving Infinite-Horizon Discounted Markov Decision Processes

Journal of the Operational Research Society ◽

10.1038/sj/jors/0431109 ◽

1992 ◽

Vol 43 (11) ◽

pp. 1095-1102

Author(s):

D J White

Keyword(s):

Markov Decision Processes ◽

Infinite Horizon ◽

Decision Processes ◽

Markov Decision

Download Full-text

A Convex Programming Approach for Discrete-Time Markov Decision Processes under the Expected Total Reward Criterion

SIAM Journal on Control and Optimization ◽

10.1137/19m1255811 ◽

2020 ◽

Vol 58 (4) ◽

pp. 2535-2566

Author(s):

François Dufour ◽

Alexandre Genadot

Keyword(s):

Convex Programming ◽

Markov Decision Processes ◽

Discrete Time ◽

Decision Processes ◽

Programming Approach ◽

Total Reward ◽

Markov Decision ◽

Reward Criterion

Download Full-text

Extreme-point solutions in Markov decision processes

Journal of Applied Probability ◽

10.1017/s002190020002413x ◽

1983 ◽

Vol 20 (04) ◽

pp. 835-842

Author(s):

David Assaf

Keyword(s):

Convex Function ◽

Extreme Point ◽

Markov Decision Processes ◽

Convex Functions ◽

Sufficient Conditions ◽

Decision Processes ◽

Markov Decision ◽

Full Solution

The paper presents sufficient conditions for certain functions to be convex. Functions of this type often appear in Markov decision processes, where their maximum is the solution of the problem. Since a convex function takes its maximum at an extreme point, the conditions may greatly simplify a problem. In some cases a full solution may be obtained after the reduction is made. Some illustrative examples are discussed.

Download Full-text