scholarly journals Reinforcement Learning Approach to Design Practical Adaptive Control for a Small-Scale Intelligent Vehicle

Symmetry ◽  
2019 ◽  
Vol 11 (9) ◽  
pp. 1139 ◽  
Author(s):  
Bo Hu ◽  
Jiaxi Li ◽  
Jie Yang ◽  
Haitao Bai ◽  
Shuang Li ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Adaptive Cruise Control ◽  
Computational Time ◽  
Small Scale ◽  
Intelligent Vehicle ◽  
Vehicle Speed ◽  
Cruise Control ◽  
Q Learning ◽  
Model Free ◽  
Control Actions

Reinforcement learning (RL) based techniques have been employed for the tracking and adaptive cruise control of a small-scale vehicle with the aim to transfer the obtained knowledge to a full-scale intelligent vehicle in the near future. Unlike most other control techniques, the purpose of this study is to seek a practical method that enables the vehicle, in the real environment and in real time, to learn the control behavior on its own while adapting to the changing circumstances. In this context, it is necessary to design an algorithm that symmetrically considers both time efficiency and accuracy. Meanwhile, in order to realize adaptive cruise control specifically, a set of symmetrical control actions consisting of steering angle and vehicle speed needs to be optimized simultaneously. In this paper, firstly, the experimental setup of the small-scale intelligent vehicle is introduced. Subsequently, three model-free RL algorithm are conducted to develop and finally form the strategy to keep the vehicle within its lanes at constant and top velocity. Furthermore, a model-based RL strategy is compared that incorporates learning from real experience and planning from simulated experience. Finally, a Q-learning based adaptive cruise control strategy is intermixed to the existing tracking control architecture to allow the vehicle slow-down in the curve and accelerate on straightaways. The experimental results show that the Q-learning and Sarsa (λ) algorithms can achieve a better tracking behavior than the conventional Sarsa, and Q-learning outperform Sarsa (λ) in terms of computational complexity. The Dyna-Q method performs similarly with the Sarsa (λ) algorithms, but with a significant reduction of computational time. Compared with a fine-tuned proportion integration differentiation (PID) controller, the good-balanced Q-learning is seen to perform better and it can also be easily applied to control problems with over one control actions.

Safe Deep Reinforcement Learning for Adaptive Cruise Control by Imposing State-Specific Safe Sets

2021 ◽  
Author(s):  
Mathis Brosowsky ◽  
Florian Keck ◽  
Jakob Ketterer ◽  
Simon Isele ◽  
Daniel Slieter ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Adaptive Cruise Control ◽  
Cruise Control

Energy Optimization of Solar Micro-Grid Using Multi Agent Reinforcement Learning

2015 ◽  
Vol 787 ◽  
pp. 843-847
Author(s):  
Leo Raju ◽  
R.S. Milton ◽  
S. Sakthiyanandan
Keyword(s):  
Reinforcement Learning ◽  
Energy Savings ◽  
Learning Method ◽  
Solar Pv ◽  
Q Learning ◽  
Pv Systems ◽  
Model Free ◽  
Individual Unit ◽  
Multi Agent ◽  
Micro Grid

In this paper, two solar Photovoltaic (PV) systems are considered; one in the department with capacity of 100 kW and the other in the hostel with capacity of 200 kW. Each one has battery and load. The capital cost and energy savings by conventional methods are compared and it is proved that the energy dependency from grid is reduced in solar micro-grid element, operating in distributed environment. In the smart grid frame work, the grid energy consumption is further reduced by optimal scheduling of the battery, using Reinforcement Learning. Individual unit optimization is done by a model free reinforcement learning method, called Q-Learning and it is compared with distributed operations of solar micro-grid using a Multi Agent Reinforcement Learning method, called Joint Q-Learning. The energy planning is designed according to the prediction of solar PV energy production and observed load pattern of department and the hostel. A simulation model was developed using Python programming.

Computational Reinforcement Learning

2015 ◽  
Author(s):  
Todd M. Gureckis ◽  
Bradley C. Love
Keyword(s):  
Reinforcement Learning ◽  
Instrumental Learning ◽  
Learning Behavior ◽  
Q Learning ◽  
Mathematical Ideas ◽  
Model Free ◽  
Versus Model ◽  
Open Questions ◽  
History Of ◽  
Exploration Exploitation

Reinforcement learning (RL) refers to the scientific study of how animals and machines adapt their behavior in order to maximize reward. The history of RL research can be traced to early work in psychology on instrumental learning behavior. However, the modern field of RL is a highly interdisciplinary area that lies that the intersection of ideas in computer science, machine learning, psychology, and neuroscience. This chapter summarizes the key mathematical ideas underlying this field including the exploration/exploitation dilemma, temporal-difference (TD) learning, Q-learning, and model-based versus model-free learning. In addition, a broad survey of open questions in psychology and neuroscience are reviewed.

scholarly journals The Effects of Vehicle Automation on Driver Engagement: The Case of Adaptive Cruise Control and Mind Wandering

2020 ◽  
pp. 001872082097485
Author(s):  
Starla M. Weaver ◽  
Stephanie M. Roldan ◽  
Tracy B. Gonzalez ◽  
Stacy A. Balk ◽  
Brian H. Philips
Keyword(s):  
Adaptive Cruise Control ◽  
Field Research ◽  
Physiological Arousal ◽  
Mind Wandering ◽  
Driving Performance ◽  
Vehicle Speed ◽  
Cruise Control ◽  
Positive Effects ◽  
And Gender ◽  
Vehicle Automation

Objective This field study examined the effects of adaptive cruise control (ACC) on mind wandering prevalence. Background ACC relieves the driver of the need to regulate vehicle speed and following distance, which may result in safety benefits. However, if ACC reduces the amount of attentional resources drivers must devote to driving, then drivers who use ACC may experience increased periods of mind wandering, which could reduce safety. Methods Participants drove a prescribed route on a public road twice, once using ACC and once driving manually. Mind wandering rates were assessed throughout the drive using auditory probes, which occurred at random intervals and required the participant to indicate whether or not they were mind wandering. Measures of physiological arousal and driving performance were also recorded. Results No evidence of increased mind wandering was found when drivers used ACC. In fact, female drivers reported reduced rates of mind wandering when driving with ACC relative to manual driving. Driving with ACC also tended to be associated with increased physiological arousal and improved driving behavior. Conclusion Use of ACC did not encourage increased mind wandering or negatively affect driving performance. In fact, the results indicate that ACC may have positive effects on driver safety among drivers who have limited experience with the technology. Application Driver characteristics, such as level of experience with in-vehicle technology and gender, should be considered when investigating driver engagement during ACC use. Field research on vehicle automation may provide valuable insights over and above studies conducted in driving simulators.

scholarly journals Battery Electric Vehicle Eco-Cooperative Adaptive Cruise Control in the Vicinity of Signalized Intersections

Energies ◽  
2020 ◽  
Vol 13 (10) ◽  
pp. 2433
Author(s):  
Hao Chen ◽  
Hesham A. Rakha
Keyword(s):  
Energy Consumption ◽  
Adaptive Cruise Control ◽  
Combustion Engine ◽  
Signalized Intersections ◽  
Simulation Software ◽  
Vehicle Speed ◽  
Cruise Control ◽  
Optimal Speed ◽  
Cooperative Adaptive Cruise Control ◽  
The Impact

This study develops a connected eco-driving controller for battery electric vehicles (BEVs), the BEV Eco-Cooperative Adaptive Cruise Control at Intersections (Eco-CACC-I). The developed controller can assist BEVs while traversing signalized intersections with minimal energy consumption. The calculation of the optimal vehicle trajectory is formulated as an optimization problem under the constraints of (1) vehicle acceleration/deceleration behavior, defined by a vehicle dynamics model; (2) vehicle energy consumption behavior, defined by a BEV energy consumption model; and (3) the relationship between vehicle speed, location, and signal timing, defined by vehicle characteristics and signal phase and timing (SPaT) data shared under a connected vehicle environment. The optimal speed trajectory is computed in real-time by the proposed BEV eco-CACC-I controller, so that a BEV can follow the optimal speed while negotiating a signalized intersection. The proposed BEV controller was tested in a case study to investigate its performance under various speed limits, roadway grades, and signal timings. In addition, a comparison of the optimal speed trajectories for BEVs and internal combustion engine vehicles (ICEVs) was conducted to investigate the impact of vehicle engine types on eco-driving solutions. Lastly, the proposed controller was implemented in microscopic traffic simulation software to test its networkwide performance. The test results from an arterial corridor with three signalized intersections demonstrate that the proposed controller can effectively reduce stop-and-go traffic in the vicinity of signalized intersections and that the BEV Eco-CACC-I controller produces average savings of 9.3% in energy consumption and 3.9% in vehicle delays.

Safe Adaptive Cruise Control with Control Barrier Function and Smith Predictor

2020 ◽  
Author(s):  
Caio I. G. Chinelato ◽  
Bruno A. Angélico
Keyword(s):  
Numerical Simulations ◽  
Barrier Function ◽  
Adaptive Cruise Control ◽  
Smith Predictor ◽  
Vehicle Speed ◽  
Cruise Control ◽  
Control Lyapunov Function ◽  
And Control ◽  
Safe Set ◽  
Initial Results

This work presents the development of Adaptive Cruise Control (ACC) applied to a vehicle. The ACC tracks a predefined controlled vehicle cruise speed, however when a leading vehicle with lower speed is encountered, the ACC must adapt the controlled vehicle speed to maintain a safe distance between the vehicles. The control strategy applied combines Control Lyapunov Function (CLF), related to performance/stability objectives and Control Barrier Function (CBF), related to safety conditions represented by a safe set. CLF and CBF are integrated with Quadratic Programming (QP) and a relaxation is used to make performance/stability objectives as a soft constraint and safety conditions as a hard constraint. The system model is based on a vehicle available at EPUSP and presents an input time-delay, that can degrade performance and stability. The input delay is compensated with a Smith Predictor. The initial results were obtained through numerical simulations and, in the future, the scheme will be implemented in the vehicle. The numerical simulations indicate that the proposed controller respect the performance/stability objectives and the safety conditions.

Sign in / Sign up

Export Citation Format

Share Document