Exploring Q-Learning Optimization in Traffic Signal Timing Plan Management

For the purpose of improving the efficiency of traffic signal control for isolate intersection under oversaturated conditions, a multi-objective optimization algorithm for traffic signal control is proposed. Throughput maximum and average queue ratio minimum are selected as the optimization objectives of the traffic signal control under oversaturated condition. A simulation environment using VISSIM SCAPI was utilized to evaluate the convergence and the optimization results under various settings and traffic conditions. It is written by C++/CRL to connect the simulation software VISSIM and the proposed algorithm. The simulation results indicated that the signal timing plan generated by the proposed algorithm has good efficiency in managing the traffic flow at oversaturated intersection than the commonly utilized signal timing optimization software Synchro. The update frequency applied in the simulation environment was 120 s, and it can meet the requirements of signal timing plan update in real filed. Thus, the proposed algorithm has the capability of searching Pareto front of the multi-objective problem domain under both normal condition and over-saturated condition.

Download Full-text

Analytical derivation of the optimal traffic signal timing: Minimizing delay variability and spillback probability for undersaturated intersections

Transportation Research Part B Methodological ◽

10.1016/j.trb.2018.11.004 ◽

2019 ◽

Vol 119 ◽

pp. 45-68 ◽

Cited By ~ 7

Author(s):

Reza Mohajerpoor ◽

Meead Saberi ◽

Mohsen Ramezani

Keyword(s):

Traffic Signal ◽

Signal Timing ◽

Analytical Derivation ◽

Delay Variability ◽

Traffic Signal Timing

Download Full-text

Use of Big Data to Evaluate and Improve Performance of Traffic Signal Systems in Resource-Constrained Countries

Transportation Research Record Journal of the Transportation Research Board ◽

10.3141/2620-03 ◽

2017 ◽

Vol 2620 (1) ◽

pp. 20-30 ◽

Cited By ~ 3

Author(s):

Yang Carl Lu ◽

Holly Krambeck ◽

Liang Tang

Keyword(s):

Traffic Control ◽

Time Of Day ◽

Traffic Signal ◽

Signal Timing ◽

Gps Data ◽

Resource Constrained ◽

Physical Sensors ◽

Constrained Environments ◽

Traffic Signal Timing ◽

High Level

Deployment of an adaptive area traffic control system is expensive; physical sensors require installation, calibration, and regular maintenance. Because of the high level of technical and financial resources required, area traffic control systems found in developing countries often are minimally functioning. In Cebu City, Philippines, for example, the Sydney Coordinated Adaptive Traffic System was installed before 2000, and fewer than 35% of detectors were still functioning as of January 2015. To address this challenge, a study was designed to determine whether taxi company GPS data are sufficient to evaluate and improve traffic signal timing plans in resource-constrained environments. If this work is successful, the number of physical sensors required to support those systems may be reduced and thereby substantially lower the costs of installation and maintenance. Taxi GPS data provided by a regional taxi-hailing app were used to design and implement methodologies for evaluating the performance of traffic signal timing plans and for deriving updated fixed-dynamic plans, which are fixed plans (with periods based on observable congestion patterns rather than only time of day) iterated regularly until optimization is reached. To date, three rounds of iterations have been conducted to ensure the stability of the proposed signal timings. Results of exploratory analysis indicate that the algorithm is capable of generating reasonable green time splits, but cycle length adjustment must be considered in the future.

Download Full-text

Multi-Agent Reinforcement Learning for Optimizing Traffic Signal Timing

10.5121/csit.2021.110102 ◽

2021 ◽

Author(s):

Areej Salaymeh ◽

Loren Schwiebert ◽

Stephen Remias

Keyword(s):

Reinforcement Learning ◽

Traffic Signals ◽

Transportation Systems ◽

Traffic Signal ◽

Urban Traffic ◽

Signal Timing ◽

Model Free ◽

Proposed Model ◽

Multi Agent ◽

Traffic Signal Timing

Designing efficient transportation systems is crucial to save time and money for drivers and for the economy as whole. One of the most important components of traffic systems are traffic signals. Currently, most traffic signal systems are configured using fixed timing plans, which are based on limited vehicle count data. Past research has introduced and designed intelligent traffic signals; however, machine learning and deep learning have only recently been used in systems that aim to optimize the timing of traffic signals in order to reduce travel time. A very promising field in Artificial Intelligence is Reinforcement Learning. Reinforcement learning (RL) is a data driven method that has shown promising results in optimizing traffic signal timing plans to reduce traffic congestion. However, model-based and centralized methods are impractical here due to the high dimensional state-action space in complex urban traffic network. In this paper, a model-free approach is used to optimize signal timing for complicated multiple four-phase signalized intersections. We propose a multi-agent deep reinforcement learning framework that aims to optimize traffic flow using data within traffic signal intersections and data coming from other intersections in a Multi-Agent Environment in what is called Multi-Agent Reinforcement Learning (MARL). The proposed model consists of state-of-art techniques such as Double Deep Q-Network and Hindsight Experience Replay (HER). This research uses HER to allow our framework to quickly learn on sparse reward settings. We tested and evaluated our proposed model via a Simulation of Urban MObility simulation (SUMO). Our results show that the proposed method is effective in reducing congestion in both peak and off-peak times.

Download Full-text