scholarly journals Deep Reinforcement Learning-Based Traffic Signal Control Using High-Resolution Event-Based Data

Entropy ◽  
2019 ◽  
Vol 21 (8) ◽  
pp. 744 ◽  
Author(s):  
Song Wang ◽  
Xu Xie ◽  
Kedi Huang ◽  
Junjie Zeng ◽  
Zimin Cai
Keyword(s):  
Reinforcement Learning ◽  
High Resolution ◽  
Control Strategy ◽  
Control Method ◽  
Control Strategies ◽  
Fixed Time ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Event Based

Reinforcement learning (RL)-based traffic signal control has been proven to have great potential in alleviating traffic congestion. The state definition, which is a key element in RL-based traffic signal control, plays a vital role. However, the data used for state definition in the literature are either coarse or difficult to measure directly using the prevailing detection systems for signal control. This paper proposes a deep reinforcement learning-based traffic signal control method which uses high-resolution event-based data, aiming to achieve cost-effective and efficient adaptive traffic signal control. High-resolution event-based data, which records the time when each vehicle-detector actuation/de-actuation event occurs, is informative and can be collected directly from vehicle-actuated detectors (e.g., inductive loops) with current technologies. Given the event-based data, deep learning techniques are employed to automatically extract useful features for traffic signal control. The proposed method is benchmarked with two commonly used traffic signal control strategies, i.e., the fixed-time control strategy and the actuated control strategy, and experimental results reveal that the proposed method significantly outperforms the commonly used control strategies.

scholarly journals Deep Q-network-based traffic signal control models

PLoS ONE ◽  
2021 ◽  
Vol 16 (9) ◽  
pp. e0256405
Author(s):  
Sangmin Park ◽  
Eum Han ◽  
Sungho Park ◽  
Harim Jeong ◽  
Ilsoo Yun
Keyword(s):  
Reinforcement Learning ◽  
Urban Areas ◽  
Control Method ◽  
Fixed Time ◽  
Traffic Signal ◽  
Time Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Control Models ◽  
Isolated Intersection

Traffic congestion has become common in urban areas worldwide. To solve this problem, the method of searching a solution using artificial intelligence has recently attracted widespread attention because it can solve complex problems such as traffic signal control. This study developed two traffic signal control models using reinforcement learning and a microscopic simulation-based evaluation for an isolated intersection and two coordinated intersections. To develop these models, a deep Q-network (DQN) was used, which is a promising reinforcement learning algorithm. The performance was evaluated by comparing the developed traffic signal control models in this research with the fixed-time signal optimized by Synchro model, which is a traffic signal optimization model. The evaluation showed that the developed traffic signal control model of the isolated intersection was validated, and the coordination of intersections was superior to that of the fixed-time signal control method.

scholarly journals An Edge Based Multi-Agent Auto Communication Method for Traffic Light Control

Sensors ◽  
2020 ◽  
Vol 20 (15) ◽  
pp. 4291 ◽  
Author(s):  
Qiang Wu ◽  
Jianqing Wu ◽  
Jun Shen ◽  
Binbin Yong ◽  
Qingguo Zhou
Keyword(s):  
Reinforcement Learning ◽  
Control Method ◽  
Communication Protocol ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Light Control ◽  
Traffic Light ◽  
Traffic Light Control ◽  
Multi Agent

With smart city infrastructures growing, the Internet of Things (IoT) has been widely used in the intelligent transportation systems (ITS). The traditional adaptive traffic signal control method based on reinforcement learning (RL) has expanded from one intersection to multiple intersections. In this paper, we propose a multi-agent auto communication (MAAC) algorithm, which is an innovative adaptive global traffic light control method based on multi-agent reinforcement learning (MARL) and an auto communication protocol in edge computing architecture. The MAAC algorithm combines multi-agent auto communication protocol with MARL, allowing an agent to communicate the learned strategies with others for achieving global optimization in traffic signal control. In addition, we present a practicable edge computing architecture for industrial deployment on IoT, considering the limitations of the capabilities of network transmission bandwidth. We demonstrate that our algorithm outperforms other methods over 17% in experiments in a real traffic simulation environment.

scholarly journals Self-organisation in traffic signal control algorithms under light traffic conditions

ORiON ◽  
2019 ◽  
Vol 35 (1) ◽  
pp. 57-87
Author(s):  
SJ Movius ◽  
JH Van Vuuren
Keyword(s):  
Simulation Model ◽  
Traffic Simulation ◽  
Control Method ◽  
Fixed Time ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Light Traffic ◽  
Traffic Conditions ◽  
Self Organisation

Fixed-time control and vehicle-actuated control are two distinct types of traffic signal control. The latter control method involves switching traffic signals based on detected traffic flows and thus offers more flexibility (appropriate for lighter traffic conditions) than the former, which relies solely on cyclic, predetermined signal phases that are better suited for heavier traffic conditions. The notion of self-organisation has relatively recently been proposed as an alternative approach towards improving traffic signal control, particularly under light traffic conditions, due to its flexible nature and its potential to result in emergent behaviour. The effectiveness of five existing self-organising traffic signal control strategies from the literature and a fixed-control strategy are compared in this paper within a newly designed agent-based, microscopic traffic simulation model. Various shortcomings of three of these algorithms are identified and algorithmic improvements are suggested to remedy these deficiencies. The relative performance improvements resulting from these algorithmic modifications are then quantified by their implementation in the aforementioned traffic simulation model. Finally, a new self-organising algorithm is proposed that is particularly effective under lighter traffic conditions.

scholarly journals A real-time signal control strategy at an isolated pedestrian crossing based on radar data

2019 ◽  
Vol 11 (3) ◽  
pp. 168781401982590 ◽  
Author(s):  
Xu Qu ◽  
Tangyi Guo ◽  
Jin Guo ◽  
Yi Lin ◽  
Bin Ran
Keyword(s):  
Real Time ◽  
Control Strategy ◽  
Fixed Time ◽  
Radar Data ◽  
Traffic Signal ◽  
Time Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Traffic Delay ◽  
Pedestrian Crossing

Fixed-time traffic signal control strategy in an isolated pedestrian crossing tends to reduce traffic capacity and expose vulnerable road users to more danger. To mitigate the negative impact of previous control strategy, this study proposed an optimal real-time signal timing strategy to protect pedestrian crossing and at the same time minimize the system-wide traffic delay. With the application of a wide-area radar data, the features of vehicles, pedestrians, and the passing time of non-motor vehicles and pedestrian were captured considering conflicts and traffic delay. The support vector machine for regression was utilized to hypothesize traffic delay by training. The discrete values of hypothetical passing time will be tested. The minimum value of delay can be recognized and the corresponding hypothetical passing time will be recommended as the green time for crossing. The performance of the proposed ORSTS outperformed the fixed-time traffic signal control strategy in reducing traffic delay by 22.3%.

scholarly journals Research on Urban Traffic Signal Control Systems Based on Cyber Physical Systems

2020 ◽  
Vol 2020 ◽  
pp. 1-16
Author(s):  
Li-li Zhang ◽  
Qi Zhao ◽  
Li Wang ◽  
Ling-yu Zhang
Keyword(s):  
Real Time ◽  
Control Strategy ◽  
Road Traffic ◽  
Control Strategies ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
General Control ◽  
Urban Road ◽  
New System

In this paper, we present a traffic cyber physical system for urban road traffic signal control, which is referred to as UTSC-CPS. With this proposed system, managers and researchers can realize the construction and simulation of various types of traffic scenarios, the rapid development, and optimization of new control strategies and can apply effective control strategies to actual traffic management. The advantages of this new system include the following. Firstly, the fusion architecture of private cloud computing and edge computing is proposed for the first time, which effectively improves the performance of software and hardware of the urban road traffic signal control system and realizes information security perception and protection in cloud and equipment, respectively, within the fusion framework; secondly, using the concept of parallel system, the depth of real-time traffic control subsystem and real-time simulation subsystem is realized. Thirdly, the idea of virtual scene basic engine and strategy agent engine is put forward in the system design, which separates data from control strategy by designing a general control strategy API and helps researchers focus on control algorithm itself without paying attention to detection data and basic data. Finally, considering China, the system designs a general control strategy API to separate data from control strategy. Most of the popular communication protocols between signal controllers and detectors are private protocols. The standard protocol conversion middleware is skillfully designed, which decouples the field equipment from the system software and achieves the universality and reliability of the control strategy. To further demonstrate the advantages of the new system, we have carried out a one-year practical test in Weifang City, Shandong Province, China. The system has been proved in terms of stability, security, scalability, practicability and rapid practice, and verification of the new control strategy. At the same time, it proves the superiority of the simulation subsystem in the performance and simulation scale by comparing the different-scale road networks of Shunyi District in Beijing and Weifang City in Shandong Province. Further tests were conducted using real intersections, and the results were equally valid.

scholarly journals Continuous residual reinforcement learning for traffic signal control optimization

2018 ◽  
Vol 45 (8) ◽  
pp. 690-702 ◽  
Author(s):  
Mohammad Aslani ◽  
Stefan Seipel ◽  
Marco Wiering
Keyword(s):  
Reinforcement Learning ◽  
Fixed Time ◽  
Traffic Signal ◽  
Signal Control ◽  
Traffic Signal Control ◽  
Microscopic Traffic Simulation ◽  
Control Optimization ◽  
Large State Space ◽  
Average Travel Time ◽  
Control Traffic

Traffic signal control can be naturally regarded as a reinforcement learning problem. Unfortunately, it is one of the most difficult classes of reinforcement learning problems owing to its large state space. A straightforward approach to address this challenge is to control traffic signals based on continuous reinforcement learning. Although they have been successful in traffic signal control, they may become unstable and fail to converge to near-optimal solutions. We develop adaptive traffic signal controllers based on continuous residual reinforcement learning (CRL-TSC) that is more stable. The effect of three feature functions is empirically investigated in a microscopic traffic simulation. Furthermore, the effects of departing streets, more actions, and the use of the spatial distribution of the vehicles on the performance of CRL-TSCs are assessed. The results show that the best setup of the CRL-TSC leads to saving average travel time by 15% in comparison to an optimized fixed-time controller.

Sign in / Sign up

Export Citation Format

Share Document