Q-learning for continuous-time linear systems: A model-free infinite horizon optimal control approach

2017 ◽  
Vol 100 ◽  
pp. 14-20 ◽  
Author(s):  
Kyriakos G. Vamvoudakis
Keyword(s):  
Optimal Control ◽  
Linear Systems ◽  
Continuous Time ◽  
Infinite Horizon ◽  
Control Approach ◽  
Q Learning ◽  
Model Free ◽  
Time Linear ◽  
Optimal Control Approach

scholarly journals Improved Q-Learning Method for Linear Discrete-Time Systems

Processes ◽  
2020 ◽  
Vol 8 (3) ◽  
pp. 368
Author(s):  
Jian Chen ◽  
Jinhua Wang ◽  
Jie Huang
Keyword(s):  
Optimal Control ◽  
Linear Systems ◽  
Discrete Time ◽  
Control Method ◽  
Learning Method ◽  
Control Laws ◽  
Q Learning ◽  
Model Free ◽  
Quadratic Optimal Control ◽  
Time Linear

In this paper, the Q-learning method for quadratic optimal control problem of discrete-time linear systems is reconsidered. The theoretical results prove that the quadratic optimal controller cannot be solved directly due to the linear correlation of the data sets. The following corollaries have been made: (1) The correlation of data is the key factor in the success for the calculation of quadratic optimal control laws by Q-learning method; (2) The control laws for linear systems cannot be derived directly by the existing Q-learning method; (3) For nonlinear systems, there are some doubts about the data independence of current method. Therefore, it is necessary to discuss the probability of the controllers established by the existing Q-learning method. To solve this problem, based on the ridge regression, an improved model-free Q-learning quadratic optimal control method for discrete-time linear systems is proposed in this paper. Therefore, the computation process can be implemented correctly, and the effective controller can be solved. The simulation results show that the proposed method can not only overcome the problem caused by the data correlation, but also derive proper control laws for discrete-time linear systems.

Sign in / Sign up

Export Citation Format

Share Document