Real-Time Coordinated Signal Control Through Use of Agents with Online Reinforcement Learning

2003 ◽  
Vol 1836 (1) ◽  
pp. 64-75 ◽  
Author(s):  
Min Chee Choy ◽  
Ruey Long Cheu ◽  
Dipti Srinivasan ◽  
Filippo Logi
Keyword(s):  
Reinforcement Learning ◽  
Real Time ◽  
Multiagent System ◽  
Fuzzy Rule ◽  
Urban Traffic ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Rule Base ◽  
Traffic Network ◽  
Average Delay

A multiagent architecture for real-time coordinated signal control in an urban traffic network is introduced. The multiagent architecture consists of three hierarchical layers of controller agents: intersection, zone, and regional controllers. Each controller agent is implemented by applying artificial intelligence concepts, namely, fuzzy logic, neural network, and evolutionary algorithm. From the fuzzy rule base, each individual controller agent recommends an appropriate signal policy at the end of each signal phase. These policies are later processed in a policy repository before being selected and implemented into the traffic network. To handle the changing dynamics of the complex traffic processes within the network, an online reinforcement learning module is used to update the knowledge base and inference rules of the agents. This concept of a multiagent system with online reinforcement learning was implemented in a network consisting of 25 signalized intersections in a microscopic traffic simulator. Initial test results showed that the multiagent system improved average delay and total vehicle stoppage time, compared with the effects of fixed-time traffic signal control.

scholarly journals A MULTI-INTERSECTION MODEL AND SIGNAL TIMING PLAN ALGORITHM FOR URBAN TRAFFIC SIGNAL CONTROL

Transport ◽  
2014 ◽  
Vol 32 (4) ◽  
pp. 368-378 ◽  
Author(s):  
Wenbin Hu ◽  
Huan Wang ◽  
Bo Du ◽  
Liping Yan
Keyword(s):  
Real Time ◽  
Traffic Flow ◽  
Delay Time ◽  
Urban Traffic ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Signal Timing ◽  
Traffic Network ◽  
Intersection Model ◽  
Volume Algorithm

The urban traffic signal control system is complex, non-linear and non-equilibrium in real conditions. The existing methods could not satisfy the requirement of real-time and dynamic control. In order to solve these difficulties and challenges, this paper proposes a novel Multi-Intersection Model (MIM) based on Cellular Automata (CA) and a Multi-Intersection Signal Timing Plan Algorithm (MISTPA), which can reduce the delay time at each intersection and effectively alleviate the traffic pressure on each intersection in the urban traffic network. Our work is divided into several parts: (1) a multi-intersection model based on CA is defined to build the dynamic urban traffic network; (2) MISTPA is proposed, which truly reflects the real-time demand degree to green time of the traffic flow at each intersection. The MISTPA is composed Single Intersection Volume Algorithm (SIVA), Single-Lane Volume Algorithm (SLVA) and single intersection signal timing plan algorithm (SISTPA). Extensive experiments show that when the saturation is greater than 0.3, the MIM and the MISTPA achieve good performance, and can significantly reduce the vehicle delay time at each intersection. The average delay time of the traffic flow at each intersection can obviously be reduced. Finally, a practical case study demonstrates that the proposed model and the corresponding algorithm are correct and effective.

scholarly journals Urban Traffic Signal Control Based on Multiobjective Joint Optimization

2020 ◽  
Vol 2020 ◽  
pp. 1-8
Author(s):  
Yongrong Wu ◽  
Yijie Zhou ◽  
Yanming Feng ◽  
Yutian Xiao ◽  
Shaojie He ◽  
...  
Keyword(s):  
Genetic Algorithm ◽  
Simulated Annealing Algorithm ◽  
Urban Traffic ◽  
Joint Optimization ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Signal Timing ◽  
Timing Optimization ◽  
Average Delay ◽  
Annealing Algorithm

This paper proposes two algorithms for signal timing optimization of single intersections, namely, microbial genetic algorithm and simulated annealing algorithm. The basis of the optimization of these two algorithms is the original timing scheme of the SCATS, and the optimized parameters are the average delay of vehicles and the capacity. Experiments verify that these two algorithms are, respectively, improved by 67.47% and 46.88%, based on the original timing scheme.

scholarly journals A Systematic Literature Review of Researches on Urban Traffic Strategy Development

2017 ◽  
Vol 3 (1) ◽  
pp. 14
Author(s):  
Zhang Lin ◽  
Cheng Wei ◽  
Wang Wei ◽  
Li Yinan ◽  
Xiao Haochen
Keyword(s):  
Traffic Signal ◽  
Urban Traffic ◽  
Current Status ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Traffic Network ◽  
Optimization Strategy ◽  
Advantages And Disadvantages ◽  
Research Perspectives ◽  
Traffic Optimization

Abstract—With the advancement of computer science and the development of urban economy, the interest of human research on urban traffic strategy has been promoted. Number of vehicles in urban traffic network in a sharp increase, in order to solve the current status of China's traffic congestion, we hope to reduce urban vehicles greenhouse gas emissions and to reduce waiting time is a serious problem currently facing the city traffic. In order to solve this problem, it can be from two aspects. On the one hand, traffic signal control of traffic network, the other is to optimize the route of the vehicle. This paper respectively from tells the development of the traffic signal control strategy and vehicle routing process, and compares their advantages and disadvantages. The paper summarizes the urban traffic strategy and traffic optimization strategy in recent years, and systematically summarizes the present situation and existing problems of urban traffic optimization strategies at home and abroad, summarizes the development prospects of urban traffic optimization strategies, and provides the strategies for traffic optimization. In order to provide the strategy of scholars engaged in the transportation of new research perspectives and research data.

scholarly journals Evaluation and Application of Urban Traffic Signal Optimizing Control Strategy Based on Reinforcement Learning

2018 ◽  
Vol 2018 ◽  
pp. 1-9
Author(s):  
Yizhe Wang ◽  
Xiaoguang Yang ◽  
Yangdong Liu ◽  
Hailun Liang
Keyword(s):  
Reinforcement Learning ◽  
Hierarchical Control ◽  
Traffic Signal ◽  
Urban Traffic ◽  
Learning Ability ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Multidimensional Space ◽  
High Flows ◽  
Multiple Intersections

Reinforcement learning method has a self-learning ability in complex multidimensional space because it does not need accurate mathematical model and due to the low requirement for prior knowledge of the environment. The single intersection, arterial lines, and regional road network of a group of multiple intersections are taken as the research object on the paper. Based on the three key parameters of cycle, arterial coordination offset, and green split, a set of hierarchical control algorithms based on reinforcement learning is constructed to optimize and improve the current signal timing scheme. However, the traffic signal optimization strategy based on reinforcement learning is suitable for complex traffic environments (high flows and multiple intersections), and the effects of which are better than the current optimization methods in the conditions of high flows in single intersections, arteries, and regional multi-intersection. In a word, the problem of insufficient traffic signal control capability is studied, and the hierarchical control algorithm based on reinforcement learning is applied to traffic signal control, so as to provide new ideas and methods for traffic signal control theory.

scholarly journals DQN-BASED TRAFFIC SIGNAL CONTROL SYSTEMS

2021 ◽  
Vol 6 (7(57)) ◽  
pp. 16-18
Author(s):  
Ivan Vladimirovich Kondratov
Keyword(s):  
Reinforcement Learning ◽  
Control Systems ◽  
Real Time ◽  
Traffic Control ◽  
Optimization Methods ◽  
Traffic Signal ◽  
Modern World ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Traffic Signal Control Systems

Real-time adaptive traffic control is an important problem in modern world. Historically, various optimization methods have been used to build adaptive traffic signal control systems. Recently, reinforcement learning has been advanced, and various papers showed efficiency of Deep-Q-Learning (DQN) in solving traffic control problems and providing real-time adaptive control for traffic, decreasing traffic pressure and lowering average travel time for drivers. In this paper we consider the problem of traffic signal control, present the basics of reinforcement learning and review the latest results in this area.

scholarly journals Smart Infrastructure for Future Urban Mobility

AI Magazine ◽  
2020 ◽  
Vol 41 (1) ◽  
pp. 5-18
Author(s):  
Stephen Smith
Keyword(s):  
Real Time ◽  
Research Effort ◽  
Traffic Signal ◽  
Urban Traffic ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Traffic Flows ◽  
Planning And Scheduling ◽  
Online Planning ◽  
Traffic Theory

Real-time traffic signal control presents a challenging multiagent planning pro­blem, particularly in urban road networks where, unlike simpler arterial settings, there are competing dominant traffic flows that shift through the day. Further complicating matters, urban environments require attention to multimodal traffic flows (vehicles, pedestrians, bicyclists, buses) that move at different speeds and may be given different priorities. For the past several years, my research group has been developing and refining a real-time, adaptive traffic signal control system to address these challenges, referred to as scalable urban traffic control (Surtrac). Combining principles from automated planning and scheduling, multiagent systems, and traffic theory, Surtrac treats traffic signal control as a decentralized online planning process. In operation, each intersection repeatedly generates and executes (in rolling horizon fashion) signal-timing plans that optimize the movement of currently sensed approaching traffic through the intersection. Each time a new plan is produced (nominally every couple of seconds), the intersection communicates to its downstream neighbors what traffic it expects to send their way, allowing intersections to construct longer horizon plans and achieve coordinated behavior. Initial evaluation of Surtrac in the field has demonstrated significant performance improvements, and the technology is now deployed and operating in several U.S. cities. More recent work has focused on integrating real-time adaptive signal control with emerging connected vehicle technology, and exploration of the opportunities for enhanced mobility that direct vehicle (or pedestrian) to infrastructure communication can provide. Current technology development efforts center on vehicle route sharing, smart transit priority, safe intersection crossing for pedestrians with disabilities, real-time incident detection, and integrated optimization of signal control and route choice decisions. This article provides an overview of this overall research effort.

scholarly journals BACKPRESSURE BASED TRAFFIC SIGNAL CONTROL CONSIDERING CAPACITY OF DOWNSTREAM LINKS

Transport ◽  
2020 ◽  
Vol 35 (4) ◽  
pp. 347-356
Author(s):  
Shenxue Hao ◽  
Licai Yang ◽  
Yunfeng Shi ◽  
Yajuan Guo
Keyword(s):  
Queue Length ◽  
Control Algorithm ◽  
Mesh Networks ◽  
Traffic Signal ◽  
Urban Traffic ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Traffic Network ◽  
Maximum Throughput ◽  
Urban Traffic Network

Congestion is a kind of expression of instability of traffic network. Traffic signal control keeping traffic network stable can reduce the congestion of urban traffic. In order to improve the efficiency of urban traffic network, this study proposes a decentralized traffic signal control strategy based on backpressure algorithm used in Wi-Fi mesh networks for packets routing. Backpressure based traffic signal control algorithm can stabilize urban traffic network and achieve maximum throughput. Based on original backpressure algorithm, the variant parameter and penalty function are considered to balance the queue differential and capacity of downstream links in urban traffic network. For each traffic phase of intersections, phase weight is computed using queue differential and capacity of downstream links, which fixed the deficiency of infinite queue capacity in original backpressure algorithm. It is proved that the extended backpressure traffic signal control algorithm can maintain stability of urban traffic network, and also can prevent queue spillback, so as to improve performance of whole traffic network. Simulations are carried out in Vissim using Vissim COM programming interface and Visual Studio development tools. Evaluation results illuminate that it can get better performance than the backpressure algorithm just based on queue length differential in average queue length and delay of traffic network.

Sign in / Sign up

Export Citation Format

Share Document