The actor-critic algorithm as multi-time-scale stochastic approximation

Vivek S Borkar; Vijaymohan R Konda

doi:10.1007/bf02745577

The actor-critic algorithm as multi-time-scale stochastic approximation

Sadhana ◽

10.1007/bf02745577 ◽

1997 ◽

Vol 22 (4) ◽

pp. 525-543 ◽

Cited By ~ 5

Author(s):

Vivek S Borkar ◽

Vijaymohan R Konda

Keyword(s):

Stochastic Approximation ◽

Time Scale

Download Full-text

Asymptotic properties of two time-scale stochastic approximation algorithms with constant step sizes

Proceedings of the 2003 American Control Conference, 2003. ◽

10.1109/acc.2003.1240536 ◽

2004 ◽

Cited By ~ 4

Author(s):

V.B. Tadic ◽

S.P. Meyn

Keyword(s):

Approximation Algorithms ◽

Stochastic Approximation ◽

Time Scale ◽

Asymptotic Properties

Download Full-text

Convergence rate and averaging of nonlinear two-time-scale stochastic approximation algorithms

The Annals of Applied Probability ◽

10.1214/105051606000000448 ◽

2006 ◽

Vol 16 (3) ◽

pp. 1671-1702 ◽

Cited By ~ 7

Author(s):

Abdelkader Mokkadem ◽

Mariane Pelletier

Keyword(s):

Approximation Algorithms ◽

Convergence Rate ◽

Stochastic Approximation ◽

Time Scale

Download Full-text

Two time-scale stochastic approximation for constrained stochastic optimization and constrained Markov decision problems

Proceedings of the 2003 American Control Conference, 2003. ◽

10.1109/acc.2003.1242471 ◽

2004 ◽

Cited By ~ 3

Author(s):

V.B. Tadic ◽

A. Doucet ◽

S. Singh

Keyword(s):

Stochastic Optimization ◽

Stochastic Approximation ◽

Time Scale ◽

Decision Problems ◽

Markov Decision Problems ◽

Markov Decision

Download Full-text

Finite-Time Analysis and Restarting Scheme for Linear Two-Time-Scale Stochastic Approximation

SIAM Journal on Control and Optimization ◽

10.1137/20m1322649 ◽

2021 ◽

Vol 59 (4) ◽

pp. 2798-2819

Author(s):

Thinh T. Doan

Keyword(s):

Stochastic Approximation ◽

Finite Time ◽

Time Scale ◽

Time Analysis

Download Full-text

Adaptive Monte Carlo Variance Reduction with Two-time-scale Stochastic Approximation

Monte Carlo Methods and Applications ◽

10.1515/mcma.2007.010 ◽

2007 ◽

Vol 13 (3) ◽

Cited By ~ 11

Author(s):

Reiichiro Kawai

Keyword(s):

Monte Carlo ◽

Stochastic Approximation ◽

Time Scale ◽

Variance Reduction ◽

Adaptive Monte Carlo

Download Full-text

On the robustness of two time-scale stochastic approximation algorithms

2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601) ◽

10.1109/cdc.2004.1429656 ◽

2004 ◽

Author(s):

V.B. Tadic

Keyword(s):

Approximation Algorithms ◽

Stochastic Approximation ◽

Time Scale

Download Full-text

Concentration bounds for two time scale stochastic approximation

2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton) ◽

10.1109/allerton.2018.8636078 ◽

2018 ◽

Author(s):

Vivek S. Borkar ◽

Sarath Pattathil

Keyword(s):

Stochastic Approximation ◽

Time Scale ◽

Concentration Bounds

Download Full-text

Convergence rate of linear two-time-scale stochastic approximation

The Annals of Applied Probability ◽

10.1214/105051604000000116 ◽

2004 ◽

Vol 14 (2) ◽

pp. 796-819 ◽

Cited By ~ 32

Author(s):

Vijay R. Konda ◽

John N. Tsitsiklis

Keyword(s):

Convergence Rate ◽

Stochastic Approximation ◽

Time Scale

Download Full-text

Linear Two-Time-Scale Stochastic Approximation A Finite-Time Analysis

2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton) ◽

10.1109/allerton.2019.8919880 ◽

2019 ◽

Author(s):

Thinh T. Doan ◽

Justin Romberg

Keyword(s):

Stochastic Approximation ◽

Finite Time ◽

Time Scale ◽

Time Analysis

Download Full-text

Two Time-Scale Stochastic Approximation with Controlled Markov Noise and Off-Policy Temporal-Difference Learning

Mathematics of Operations Research ◽

10.1287/moor.2017.0855 ◽

2018 ◽

Vol 43 (1) ◽

pp. 130-151 ◽

Cited By ~ 3

Author(s):

Prasenjit Karmakar ◽

Shalabh Bhatnagar

Keyword(s):

Stochastic Approximation ◽

Time Scale ◽

Temporal Difference ◽

Temporal Difference Learning

Download Full-text