scholarly journals An Efficiency Enhancing Methodology for Multiple Autonomous Vehicles in an Urban Network Adopting Deep Reinforcement Learning

2021 ◽  
Vol 11 (4) ◽  
pp. 1514 ◽  
Author(s):  
Quang-Duy Tran ◽  
Sang-Hoon Bae
Keyword(s):  
Reinforcement Learning ◽  
Traffic Congestion ◽  
Autonomous Vehicles ◽  
Penetration Rate ◽  
Autonomous Vehicle ◽  
Effective Means ◽  
Urban Network ◽  
Learning Agents ◽  
Policy Optimization ◽  
The Impact

To reduce the impact of congestion, it is necessary to improve our overall understanding of the influence of the autonomous vehicle. Recently, deep reinforcement learning has become an effective means of solving complex control tasks. Accordingly, we show an advanced deep reinforcement learning that investigates how the leading autonomous vehicles affect the urban network under a mixed-traffic environment. We also suggest a set of hyperparameters for achieving better performance. Firstly, we feed a set of hyperparameters into our deep reinforcement learning agents. Secondly, we investigate the leading autonomous vehicle experiment in the urban network with different autonomous vehicle penetration rates. Thirdly, the advantage of leading autonomous vehicles is evaluated using entire manual vehicle and leading manual vehicle experiments. Finally, the proximal policy optimization with a clipped objective is compared to the proximal policy optimization with an adaptive Kullback–Leibler penalty to verify the superiority of the proposed hyperparameter. We demonstrate that full automation traffic increased the average speed 1.27 times greater compared with the entire manual vehicle experiment. Our proposed method becomes significantly more effective at a higher autonomous vehicle penetration rate. Furthermore, the leading autonomous vehicles could help to mitigate traffic congestion.

scholarly journals Platoon Signal Priority in Connected-Autonomous Vehicle Environments: Algorithm Development and Testing

2019 ◽  
Vol 65 (4) ◽  
pp. 1-9
Author(s):  
Milan Zlatkovic ◽  
Andalib Shams
Keyword(s):  
Traffic Congestion ◽  
Autonomous Vehicles ◽  
Penetration Rate ◽  
Autonomous Vehicle ◽  
Transportation Systems ◽  
Vehicle Technologies ◽  
Measures Of Effectiveness ◽  
Urban Corridor ◽  
Intersection Control ◽  
Day By Day

As traffic congestion increases day by day, it becomes necessary to improve the existing roadway facilities to maintain satisfactory operational and safety performances. New vehicle technologies, such as Connected and Autonomous Vehicles (CAV) have a potential to significantly improve transportation systems. Using the advantages of CAVs, this study developed signalized intersection control strategy algorithm that optimizes the operations of CAVs and allows signal priority for connected platoons. The algorithm was tested in VISSIM microsimulation using a real-world urban corridor. The tested scenarios include a 2040 Do-Nothing scenario, and CAV alternatives with 25%, 50%, 75% and 100% CAV penetration rate. The results show a significant reduction in intersection delays (26% - 38%) and travel times (6% - 20%), depending on the penetration rate, as well as significant improvements on the network-wide level. CAV penetration rates of 50% or more have a potential to significantly improve all operational measures of effectiveness.

scholarly journals Proximal Policy Optimization Through a Deep Reinforcement Learning Framework for Multiple Autonomous Vehicles at a Non-Signalized Intersection

2020 ◽  
Vol 10 (16) ◽  
pp. 5722 ◽  
Author(s):  
Duy Quang Tran ◽  
Sang-Hoon Bae
Keyword(s):  
Reinforcement Learning ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Signalized Intersection ◽  
Continuous Control ◽  
Simulation Performance ◽  
Perceptron Algorithm ◽  
Learning Framework ◽  
Positive Effects ◽  
Policy Optimization

Advanced deep reinforcement learning shows promise as an approach to addressing continuous control tasks, especially in mixed-autonomy traffic. In this study, we present a deep reinforcement-learning-based model that considers the effectiveness of leading autonomous vehicles in mixed-autonomy traffic at a non-signalized intersection. This model integrates the Flow framework, the simulation of urban mobility simulator, and a reinforcement learning library. We also propose a set of proximal policy optimization hyperparameters to obtain reliable simulation performance. First, the leading autonomous vehicles at the non-signalized intersection are considered with varying autonomous vehicle penetration rates that range from 10% to 100% in 10% increments. Second, the proximal policy optimization hyperparameters are input into the multiple perceptron algorithm for the leading autonomous vehicle experiment. Finally, the superiority of the proposed model is evaluated using all human-driven vehicle and leading human-driven vehicle experiments. We demonstrate that full-autonomy traffic can improve the average speed and delay time by 1.38 times and 2.55 times, respectively, compared with all human-driven vehicle experiments. Our proposed method generates more positive effects when the autonomous vehicle penetration rate increases. Additionally, the leading autonomous vehicle experiment can be used to dissipate the stop-and-go waves at a non-signalized intersection.

scholarly journals Flow: Deep Reinforcement Learning for Control in SUMO

10.29007/dkzb ◽  
2018 ◽  
Author(s):  
Nishant Kheterpal ◽  
Kanaad Parvate ◽  
Cathy Wu ◽  
Aboudy Kreidieh ◽  
Eugene Vinitsky ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Autonomous Vehicles ◽  
Computational Framework ◽  
Design Decisions ◽  
Learning Agents ◽  
Computational Overhead ◽  
Collision Handling ◽  
System Flow ◽  
Single Vehicle ◽  
Policy Optimization

We detail the motivation and design decisions underpinning Flow, a computational framework integrating SUMO with the deep reinforcement learning libraries rllab and RLlib, allowing researchers to apply deep reinforcement learning (RL) methods to traffic scenarios, and permitting vehicle and infrastructure control in highly varied traffic envi- ronments. Users of Flow can rapidly design a wide variety of traffic scenarios in SUMO, enabling the development of controllers for autonomous vehicles and intelligent infrastruc- ture across a broad range of settings.Flow facilitates the use of policy optimization algorithms to train controllers that can optimize for highly customizable traffic metrics, such as traffic flow or system-wide average velocity. Training reinforcement learning agents using such methods requires a massive amount of data, thus simulator reliability and scalability were major challenges in the development of Flow. A contribution of this work is a variety of practical techniques for overcoming such challenges with SUMO, including parallelizing policy rollouts, smart exception and collision handling, and leveraging subscriptions to reduce computational overhead.To demonstrate the resulting performance and reliability of Flow, we introduce the canonical single-lane ring road benchmark and briefly discuss prior work regarding that task. We then pose a more complex and challenging multi-lane setting and present a trained controller for a single vehicle that stabilizes the system. Flow is an open-source tool and available online at https://github.com/cathywu/flow.

Deep reinforcement learning based path tracking controller for autonomous vehicle

2020 ◽  
pp. 095440702095459
Author(s):  
I-Ming Chen ◽  
Ching-Yao Chan
Keyword(s):  
Reinforcement Learning ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Path Tracking ◽  
Steering Control ◽  
Alternative Approach ◽  
Control Command ◽  
Pure Pursuit ◽  
Driving Condition ◽  
Policy Optimization

Path tracking is an essential task for autonomous vehicles (AV), for which controllers are designed to issue commands so that the AV will follow the planned path properly to ensure operational safety, comfort, and efficiency. While solving the time-varying nonlinear vehicle dynamic problem is still challenging today, deep neural network (NN) methods, with their capability to deal with nonlinear systems, provide an alternative approach to tackle the difficulties. This study explores the potential of using deep reinforcement learning (DRL) for vehicle control and applies it to the path tracking task. In this study, proximal policy optimization (PPO) is selected as the DRL algorithm and is combined with the conventional pure pursuit (PP) method to structure the vehicle controller architecture. The PP method is used to generate a baseline steering control command, and the PPO is used to derive a correction command to mitigate the inaccuracy associated with the baseline from PP. The blend of the two controllers makes the overall operation more robust and adaptive and attains the optimality to improve tracking performance. In this paper, the structure, settings and training process of the PPO are described. Simulation experiments are carried out based on the proposed methodology, and the results show that the path tracking capability in a low-speed driving condition is significantly enhanced.

scholarly journals An Evaluation Methodology for Interactive Reinforcement Learning with Simulated Users

Biomimetics ◽  
2021 ◽  
Vol 6 (1) ◽  
pp. 13
Author(s):  
Adam Bignold ◽  
Francisco Cruz ◽  
Richard Dazeley ◽  
Peter Vamplew ◽  
Cameron Foale
Keyword(s):  
Reinforcement Learning ◽  
Information Source ◽  
Human Interaction ◽  
Evaluation Methodology ◽  
External Information ◽  
Preliminary Evaluation ◽  
Learning Agents ◽  
Learning Agent ◽  
Knowledge Bias ◽  
The Impact

Interactive reinforcement learning methods utilise an external information source to evaluate decisions and accelerate learning. Previous work has shown that human advice could significantly improve learning agents’ performance. When evaluating reinforcement learning algorithms, it is common to repeat experiments as parameters are altered or to gain a sufficient sample size. In this regard, to require human interaction every time an experiment is restarted is undesirable, particularly when the expense in doing so can be considerable. Additionally, reusing the same people for the experiment introduces bias, as they will learn the behaviour of the agent and the dynamics of the environment. This paper presents a methodology for evaluating interactive reinforcement learning agents by employing simulated users. Simulated users allow human knowledge, bias, and interaction to be simulated. The use of simulated users allows the development and testing of reinforcement learning agents, and can provide indicative results of agent performance under defined human constraints. While simulated users are no replacement for actual humans, they do offer an affordable and fast alternative for evaluative assisted agents. We introduce a method for performing a preliminary evaluation utilising simulated users to show how performance changes depending on the type of user assisting the agent. Moreover, we describe how human interaction may be simulated, and present an experiment illustrating the applicability of simulating users in evaluating agent performance when assisted by different types of trainers. Experimental results show that the use of this methodology allows for greater insight into the performance of interactive reinforcement learning agents when advised by different users. The use of simulated users with varying characteristics allows for evaluation of the impact of those characteristics on the behaviour of the learning agent.

scholarly journals Deep reinforcement learning based control for Autonomous Vehicles in CARLA

2022 ◽  
Author(s):  
Óscar Pérez-Gil ◽  
Rafael Barea ◽  
Elena López-Guillén ◽  
Luis M. Bergasa ◽  
Carlos Gómez-Huélamo ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Vehicle Control ◽  
Data Sources ◽  
Simulation Environment ◽  
Urban Simulation ◽  
Policy Gradient ◽  
Almost All ◽  
Control Layer

AbstractNowadays, Artificial Intelligence (AI) is growing by leaps and bounds in almost all fields of technology, and Autonomous Vehicles (AV) research is one more of them. This paper proposes the using of algorithms based on Deep Learning (DL) in the control layer of an autonomous vehicle. More specifically, Deep Reinforcement Learning (DRL) algorithms such as Deep Q-Network (DQN) and Deep Deterministic Policy Gradient (DDPG) are implemented in order to compare results between them. The aim of this work is to obtain a trained model, applying a DRL algorithm, able of sending control commands to the vehicle to navigate properly and efficiently following a determined route. In addition, for each of the algorithms, several agents are presented as a solution, so that each of these agents uses different data sources to achieve the vehicle control commands. For this purpose, an open-source simulator such as CARLA is used, providing to the system with the ability to perform a multitude of tests without any risk into an hyper-realistic urban simulation environment, something that is unthinkable in the real world. The results obtained show that both DQN and DDPG reach the goal, but DDPG obtains a better performance. DDPG perfoms trajectories very similar to classic controller as LQR. In both cases RMSE is lower than 0.1m following trajectories with a range 180-700m. To conclude, some conclusions and future works are commented.

scholarly journals Novel Routing Algorithm for Autonomous Vehicles in Smart Transportation System

2021 ◽  
Vol 3 (3) ◽  
pp. 164-179
Author(s):  
Haoxiang Wang
Keyword(s):  
Traffic Congestion ◽  
Autonomous Vehicles ◽  
Intelligent Transportation System ◽  
Autonomous Vehicle ◽  
Routing Algorithm ◽  
Transportation System ◽  
Traffic Condition ◽  
Smart Transportation ◽  
Centralized Management ◽  
The City

In recent times Automation is emerging every day and bloomed in every sector. Intelligent Transportation System (ITS) is one of the important branches of Automation. The major constrain in the transportation system is traffic congestion. This slurps the individual’s time and consequently pollutes the environment. A centralized management is required for optimizing the transportation system. The current traffic condition is predicted by evaluating the historical data and thereby it reduces the traffic congestion. The periodic update of traffic condition in each and every street of the city is obtained and the data is transferred to the autonomous vehicle. These data are obtained from the simulation results of transportation prediction tool SUMO. It is proved that our proposed work reduces the traffic congestion and maintains ease traffic flow and preserves the fleet management.

scholarly journals Assessment of Social Acceptance for Autonomous Vehicles in Southeastern Poland

Energies ◽  
2021 ◽  
Vol 14 (18) ◽  
pp. 5778
Author(s):  
Agnieszka Dudziak ◽  
Monika Stoma ◽  
Andrzej Kuranc ◽  
Jacek Caban
Keyword(s):  
Autonomous Vehicles ◽  
Social Acceptance ◽  
New Technologies ◽  
Consumer Preferences ◽  
Autonomous Vehicle ◽  
Developed Countries ◽  
Automotive Market ◽  
Advantages And Disadvantages ◽  
Autonomous Cars ◽  
The Impact

New technologies reaching out for meeting the needs of an aging population in developed countries have given rise to the development and gradual implementation of the concept of an autonomous vehicle (AV) and have even made it a necessity and an important business paradigm. However, in parallel, there is a discussion about consumer preferences and the willingness to pay for new car technologies and intelligent vehicle options. The main aim of the study was to analyze the impact of selected factors on the perception of the future of autonomous cars by respondents from the area of Southeastern Poland in terms of a comparison with traditional cars, with particular emphasis on the advantages and disadvantages of this concept. The research presented in this study was conducted in 2019 among a group of 579 respondents. Data analysis made it possible to identify potential advantages and disadvantages of the concept of introducing autonomous cars. A positive result of the survey is that 68% of respondents stated that AV will be gradually introduced to our market, which confirms the high acceptance of this technology by Poles. The obtained research results may be valuable information for governmental and local authorities, but also for car manufacturers and their future users. It is an important issue in the area of shaping the strategy of actions concerning further directions of development on the automotive market.

scholarly journals CONGESTION EFFECTS OF AUTONOMOUS TAXI FLEETS

Transport ◽  
2018 ◽  
Vol 33 (4) ◽  
pp. 971-980 ◽  
Author(s):  
Michal Maciejewski ◽  
Joschka Bischoff
Keyword(s):  
Traffic Congestion ◽  
Autonomous Vehicles ◽  
Large Scale ◽  
Traffic Volume ◽  
City Centre ◽  
Conventional Vehicle ◽  
Road Capacity ◽  
Private Car ◽  
The One ◽  
The Impact

Fleets of shared Autonomous Vehicles (AVs) could replace private cars by providing a taxi-like service but at a cost similar to driving a private car. On the one hand, large Autonomous Taxi (AT) fleets may result in increased road capacity and lower demand for parking spaces. On the other hand, an increase in vehicle trips is very likely, as travelling becomes more convenient and affordable, and additionally, ATs need to drive unoccupied between requests. This study evaluates the impact of a city-wide introduction of ATs on traffic congestion. The analysis is based on a multi-agent transport simulation (MATSim) of Berlin (Germany) and the neighbouring Brandenburg area. The central focus is on precise simulation of both real-time AT operation and mixed autonomous/conventional vehicle traffic flow. Different ratios of replacing private car trips with AT trips are used to estimate the possible effects at different stages of introducing such services. The obtained results suggest that large fleets operating in cities may have a positive effect on traffic if road capacity increases according to current predictions. ATs will practically eliminate traffic congestion, even in the city centre, despite the increase in traffic volume. However, given no flow capacity improvement, such services cannot be introduced on a large scale, since the induced additional traffic volume will intensify today’s congestion.

scholarly journals The Impact of Social Influence, Technophobia, and Perceived Safety on Autonomous Vehicle Technology Adoption

2019 ◽  
Vol 48 (2) ◽  
pp. 133-142
Author(s):  
Sahil Koul ◽  
Ali Eydgahi
Keyword(s):  
Technology Adoption ◽  
Social Influence ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Negative Relationship ◽  
Future Research ◽  
Intention To Use ◽  
Perceived Safety ◽  
Vehicle Technology ◽  
The Impact

The objective of this study was to determine whether there was a relationship between social influence, technophobia, perceived safety of autonomous vehicle technology, number of automobile-related accidents and the intention to use autonomous vehicles. The methodology was a descriptive, cross-sectional, correlational study. Theory of Planned Behavior provided the underlying theoretical framework. An online survey was the primary method of data collection. Pearson’s correlation and multiple linear regression were used for data analysis. This study found that both social influence and perceived safety of autonomous vehicle technology had significant, positive relationships with the intention to use autonomous vehicles. Additionally, a significant negative relationship was found among technophobia and intention to use autonomous vehicles. However, no relationship was found between the number of automobile-related accidents and intention to use autonomous vehicles. This study presents several original and significant findings as a contribution to the literature on autonomous vehicle technology adoption and proposes new dimensions of future research within this emerging field.

Sign in / Sign up

Export Citation Format

Share Document