scholarly journals The Study of Reinforcement Learning for Traffic Self-Adaptive Control under Multiagent Markov Game Environment

2013 ◽  
Vol 2013 ◽  
pp. 1-10 ◽  
Author(s):  
Lun-Hui Xu ◽  
Xin-Hai Xia ◽  
Qiang Luo
Keyword(s):  
Adaptive Control ◽  
Reinforcement Learning ◽  
Imperfect Information ◽  
Single Agent ◽  
Game Problem ◽  
Control Agent ◽  
Urban Traffic ◽  
Traffic Signal Control ◽  
Markov Game ◽  
Self Adaptive

Urban traffic self-adaptive control problem is dynamic and uncertain, so the states of traffic environment are hard to be observed. Efficient agent which controls a single intersection can be discovered automatically via multiagent reinforcement learning. However, in the majority of the previous works on this approach, each agent needed perfect observed information when interacting with the environment and learned individually with less efficient coordination. This study casts traffic self-adaptive control as a multiagent Markov game problem. The design employs traffic signal control agent (TSCA) for each signalized intersection that coordinates with neighboring TSCAs. A mathematical model for TSCAs’ interaction is built based on nonzero-sum markov game which has been applied to let TSCAs learn how to cooperate. A multiagent Markov game reinforcement learning approach is constructed on the basis of single-agentQ-learning. This method lets each TSCA learn to update itsQ-values under the joint actions and imperfect information. The convergence of the proposed algorithm is analyzed theoretically. The simulation results show that the proposed method is convergent and effective in realistic traffic self-adaptive control setting.

scholarly journals Evaluation and Application of Urban Traffic Signal Optimizing Control Strategy Based on Reinforcement Learning

2018 ◽  
Vol 2018 ◽  
pp. 1-9
Author(s):  
Yizhe Wang ◽  
Xiaoguang Yang ◽  
Yangdong Liu ◽  
Hailun Liang
Keyword(s):  
Reinforcement Learning ◽  
Hierarchical Control ◽  
Traffic Signal ◽  
Urban Traffic ◽  
Learning Ability ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Multidimensional Space ◽  
High Flows ◽  
Multiple Intersections

Reinforcement learning method has a self-learning ability in complex multidimensional space because it does not need accurate mathematical model and due to the low requirement for prior knowledge of the environment. The single intersection, arterial lines, and regional road network of a group of multiple intersections are taken as the research object on the paper. Based on the three key parameters of cycle, arterial coordination offset, and green split, a set of hierarchical control algorithms based on reinforcement learning is constructed to optimize and improve the current signal timing scheme. However, the traffic signal optimization strategy based on reinforcement learning is suitable for complex traffic environments (high flows and multiple intersections), and the effects of which are better than the current optimization methods in the conditions of high flows in single intersections, arteries, and regional multi-intersection. In a word, the problem of insufficient traffic signal control capability is studied, and the hierarchical control algorithm based on reinforcement learning is applied to traffic signal control, so as to provide new ideas and methods for traffic signal control theory.

Real-Time Coordinated Signal Control Through Use of Agents with Online Reinforcement Learning

2003 ◽  
Vol 1836 (1) ◽  
pp. 64-75 ◽  
Author(s):  
Min Chee Choy ◽  
Ruey Long Cheu ◽  
Dipti Srinivasan ◽  
Filippo Logi
Keyword(s):  
Reinforcement Learning ◽  
Real Time ◽  
Multiagent System ◽  
Fuzzy Rule ◽  
Urban Traffic ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Rule Base ◽  
Traffic Network ◽  
Average Delay

A multiagent architecture for real-time coordinated signal control in an urban traffic network is introduced. The multiagent architecture consists of three hierarchical layers of controller agents: intersection, zone, and regional controllers. Each controller agent is implemented by applying artificial intelligence concepts, namely, fuzzy logic, neural network, and evolutionary algorithm. From the fuzzy rule base, each individual controller agent recommends an appropriate signal policy at the end of each signal phase. These policies are later processed in a policy repository before being selected and implemented into the traffic network. To handle the changing dynamics of the complex traffic processes within the network, an online reinforcement learning module is used to update the knowledge base and inference rules of the agents. This concept of a multiagent system with online reinforcement learning was implemented in a network consisting of 25 signalized intersections in a microscopic traffic simulator. Initial test results showed that the multiagent system improved average delay and total vehicle stoppage time, compared with the effects of fixed-time traffic signal control.

scholarly journals A Review of the Self-Adaptive Traffic Signal Control System Based on Future Traffic Environment

2018 ◽  
Vol 2018 ◽  
pp. 1-12 ◽  
Author(s):  
Yizhe Wang ◽  
Xiaoguang Yang ◽  
Hailun Liang ◽  
Yangdong Liu
Keyword(s):  
Adaptive Control ◽  
Control System ◽  
Control Method ◽  
Autonomous Driving ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Control Methods ◽  
Adaptive Traffic Signal Control ◽  
Self Learning ◽  
Self Adaptive

The self-adaptive traffic signal control system serves as an effective measure for relieving urban traffic congestion. The system is capable of adjusting the signal timing parameters in real time according to the seasonal changes and short-term fluctuation of traffic demand, resulting in improvement of the efficiency of traffic operation on urban road networks. The development of information technologies on computing science, autonomous driving, vehicle-to-vehicle, and mobile Internet has created a sufficient abundance of acquisition means for traffic data. Great improvements for data acquisition include the increase of available amount of holographic data, available data types, and accuracy. The article investigates the development of commonly used self-adaptive signal control systems in the world, their technical characteristics, the current research status of self-adaptive control methods, and the signal control methods for heterogeneous traffic flow composed of connected vehicles and autonomous vehicles. Finally, the article concluded that signal control based on multiagent reinforcement learning is a kind of closed-loop feedback adaptive control method, which outperforms many counterparts in terms of real-time characteristic, accuracy, and self-learning and therefore will be an important research focus of control method in future due to the property of “model-free” and “self-learning” that well accommodates the abundance of traffic information data. Besides, it will also provide an entry point and technical support for the development of Vehicle-to-X systems, Internet of vehicles, and autonomous driving industries. Therefore, the related achievements of the adaptive control system for the future traffic environment have extremely broad application prospects.

Adaptive Traffic Signal Control Based on Path Priority

2015 ◽  
Vol 727-728 ◽  
pp. 675-682
Author(s):  
Jian Wei Zhang ◽  
Guo Zhen Tan ◽  
Nan Ding ◽  
Ming Jian Liu
Keyword(s):  
Adaptive Control ◽  
Traffic Congestion ◽  
Maximum Clique ◽  
Intersection Point ◽  
Urban Traffic ◽  
Maximum Clique Problem ◽  
Good Effect ◽  
Traffic Signal Control ◽  
Adaptive Traffic Signal Control ◽  
The City

in face of urban traffic congestion, the existing trafficcontrol system available today pursues rapid evacuation of vehicles at singleintersection and fails to take into account of the coordination between intersections,resulting in the worsening of urban traffic congestion. In order to solve theurban traffic congestion problem, an adaptive control based on the priority ofthe city path is proposed based on the PageRank priority model, a bettersolution to the traffic congestion of the city. In this strategy, the aim is tominimize the queue length of vehicles in urban road network. The releasepriority of each section is calculated based on the PageRank model. For asingle intersection point of control, the vehicle traffic volume is optimizedby the Maximum Clique Problem theory. The experiment takes 19 intersections inDalian as examples and is carried out on the VISSIM traffic simulation platform.The simulation results show that the adaptive control based on the pathpriority has good effect on reducing urban traffic congestion and evacuating vehicles.

Sign in / Sign up

Export Citation Format

Share Document