Continuous‐time receding‐horizon reinforcement learning and its application to path‐tracking control of autonomous ground vehicles

2021 ◽  
Author(s):  
Yang Lu ◽  
Wenzhang Li ◽  
Xinglong Zhang ◽  
Xin Xu
Keyword(s):  
Reinforcement Learning ◽  
Continuous Time ◽  
Tracking Control ◽  
Path Tracking ◽  
Ground Vehicles ◽  
Receding Horizon ◽  
Autonomous Ground Vehicles

A novel adaptive control approach for path tracking control of autonomous vehicles subject to uncertain dynamics

2020 ◽  
Vol 234 (8) ◽  
pp. 2115-2126
Author(s):  
Ardashir Mohammadzadeh ◽  
Hamid Taghavifar
Keyword(s):  
Autonomous Vehicles ◽  
Tracking Control ◽  
Control Method ◽  
Path Tracking ◽  
Operating Conditions ◽  
Ground Vehicles ◽  
Control Approach ◽  
Autonomous Ground Vehicles ◽  
Lane Changes ◽  
Uncertain Dynamics

Autonomous ground vehicles are constantly exposed to matched/mismatched uncertainties and disturbances and different operating conditions. Consequently, robustness to resist the undesirable effect of changes in the nominal parameters of the vehicle is a significant provision for satisfactory path-tracking control of these vehicles. The accomplishment of lateral path-tracking control is an essential task expectable from autonomous ground vehicles, particularly during critical maneuvers, abrupt cornering, and lane changes at high speeds. This paper presents a new control approach based on immersion and invariance control theorem. The asymptotic stability of the proposed method is ensured and the adaptation laws for the parameters are derived based on the I&I stability theorem. The effectiveness of the proposed control method is confirmed for autonomous ground vehicles systems while making a double-lane-change at various forward speeds. The robustness of the proposed control method is evaluated under parametric uncertainties related to the autonomous ground vehicle and different road conditions. The obtained results suggest that the proposed control method holds the capacity to be applied effectively to the path-tracking task of autonomous ground vehicles under a broad range of operating conditions, parametric uncertainness, and external disturbances.

scholarly journals Decentralized Reinforcement Learning Robust Optimal Tracking Control for Time Varying Constrained Reconfigurable Modular Robot Based on ACI andQ-Function

2013 ◽  
Vol 2013 ◽  
pp. 1-16 ◽  
Author(s):  
Bo Dong ◽  
Yuanchun Li
Keyword(s):  
Optimal Control ◽  
Reinforcement Learning ◽  
Control Theory ◽  
Continuous Time ◽  
Tracking Control ◽  
Control Policy ◽  
Time Varying ◽  
Optimal Tracking ◽  
Tracking Controller ◽  
Global Uncertainty

A novel decentralized reinforcement learning robust optimal tracking control theory for time varying constrained reconfigurable modular robots based on action-critic-identifier (ACI) and state-action value function (Q-function) has been presented to solve the problem of the continuous time nonlinear optimal control policy for strongly coupled uncertainty robotic system. The dynamics of time varying constrained reconfigurable modular robot is described as a synthesis of interconnected subsystem, and continuous time state equation andQ-function have been designed in this paper. Combining with ACI and RBF network, the global uncertainty of the subsystem and the HJB (Hamilton-Jacobi-Bellman) equation have been estimated, where critic-NN and action-NN are used to approximate the optimalQ-function and the optimal control policy, and the identifier is adopted to identify the global uncertainty as well as RBF-NN which is used to update the weights of ACI-NN. On this basis, a novel decentralized robust optimal tracking controller of the subsystem is proposed, so that the subsystem can track the desired trajectory and the tracking error can converge to zero in a finite time. The stability of ACI and the robust optimal tracking controller are confirmed by Lyapunov theory. Finally, comparative simulation examples are presented to illustrate the effectiveness of the proposed ACI and decentralized control theory.

scholarly journals Self-Optimizing Path Tracking Controller for Intelligent Vehicles Based on Reinforcement Learning

Symmetry ◽  
2021 ◽  
Vol 14 (1) ◽  
pp. 31
Author(s):  
Jichang Ma ◽  
Hui Xie ◽  
Kang Song ◽  
Hao Liu
Keyword(s):  
Reinforcement Learning ◽  
Pid Control ◽  
Tracking Control ◽  
Tracking Error ◽  
Path Tracking ◽  
Pid Controllers ◽  
Model Parameters ◽  
Control Parameters ◽  
Tracking Controller ◽  
Complex Trajectories

The path tracking control system is a crucial component for autonomous vehicles; it is challenging to realize accurate tracking control when approaching a wide range of uncertain situations and dynamic environments, particularly when such control must perform as well as, or better than, human drivers. While many methods provide state-of-the-art tracking performance, they tend to emphasize constant PID control parameters, calibrated by human experience, to improve tracking accuracy. A detailed analysis shows that PID controllers inefficiently reduce the lateral error under various conditions, such as complex trajectories and variable speed. In addition, intelligent driving vehicles are highly non-linear objects, and high-fidelity models are unavailable in most autonomous systems. As for the model-based controller (MPC or LQR), the complex modeling process may increase the computational burden. With that in mind, a self-optimizing, path tracking controller structure, based on reinforcement learning, is proposed. For the lateral control of the vehicle, a steering method based on the fusion of the reinforcement learning and traditional PID controllers is designed to adapt to various tracking scenarios. According to the pre-defined path geometry and the real-time status of the vehicle, the interactive learning mechanism, based on an RL framework (actor–critic—a symmetric network structure), can realize the online optimization of PID control parameters in order to better deal with the tracking error under complex trajectories and dynamic changes of vehicle model parameters. The adaptive performance of velocity changes was also considered in the tracking process. The proposed controlling approach was tested in different path tracking scenarios, both the driving simulator platforms and on-site vehicle experiments have verified the effects of our proposed self-optimizing controller. The results show that the approach can adaptively change the weights of PID to maintain a tracking error (simulation: within ±0.071 m; realistic vehicle: within ±0.272 m) and steering wheel vibration standard deviations (simulation: within ±0.04°; realistic vehicle: within ±80.69°); additionally, it can adapt to high-speed simulation scenarios (the maximum speed is above 100 km/h and the average speed through curves is 63–76 km/h).

Sign in / Sign up

Export Citation Format

Share Document