scholarly journals Online regret bounds for Markov decision processes with deterministic transitions

2010 ◽  
Vol 411 (29-30) ◽  
pp. 2684-2695 ◽  
Author(s):  
Ronald Ortner
Keyword(s):  
Markov Decision Processes ◽  
Decision Processes ◽  
Markov Decision ◽  
Regret Bounds

scholarly journals Regret Bounds for Reinforcement Learning via Markov Chain Concentration

2020 ◽  
Vol 67 ◽  
pp. 115-128
Author(s):  
Ronald Ortner
Keyword(s):  
Markov Chain ◽  
Reinforcement Learning ◽  
Markov Decision Processes ◽  
Mixing Time ◽  
Decision Processes ◽  
Time Parameter ◽  
Markov Decision ◽  
Regret Bounds ◽  
Optimistic Algorithm ◽  
S States

We give a simple optimistic algorithm for which it is easy to derive regret bounds of O(sqrt{t-mix SAT}) steps in uniformly ergodic Markov decision processes with S states, A actions, and mixing time parameter t-mix. These bounds are the first regret bounds in the general, non-episodic setting with an optimal dependence on all given parameters. They could only be improved by using an alternative mixing time parameter.

Extreme-point solutions in Markov decision processes

1983 ◽  
Vol 20 (04) ◽  
pp. 835-842
Author(s):  
David Assaf
Keyword(s):  
Convex Function ◽  
Extreme Point ◽  
Markov Decision Processes ◽  
Convex Functions ◽  
Sufficient Conditions ◽  
Decision Processes ◽  
Markov Decision ◽  
Full Solution

The paper presents sufficient conditions for certain functions to be convex. Functions of this type often appear in Markov decision processes, where their maximum is the solution of the problem. Since a convex function takes its maximum at an extreme point, the conditions may greatly simplify a problem. In some cases a full solution may be obtained after the reduction is made. Some illustrative examples are discussed.

Sign in / Sign up

Export Citation Format

Share Document