Informed Monte Carlo Tree Search for Real-Time Strategy games

2016 ◽  
Author(s):  
Santiago Ontanon
Keyword(s):  
Monte Carlo ◽  
Real Time ◽  
Tree Search ◽  
Monte Carlo Tree Search ◽  
Strategy Games

scholarly journals Real-Time Monte Carlo Tree Search in Ms Pac-Man

2014 ◽  
Vol 6 (3) ◽  
pp. 245-257 ◽  
Author(s):  
Tom Pepels ◽  
Mark H. M. Winands ◽  
Marc Lanctot
Keyword(s):  
Monte Carlo ◽  
Real Time ◽  
Tree Search ◽  
Monte Carlo Tree Search

Monte Carlo Tree Search and Cognitive Hierarchy Theory for Interactive-Behavior Prediction in Fast Trajectory Planning and Automated Lane Change

2021 ◽  
Vol 1 (1) ◽  
Author(s):  
Shahab Karimi ◽  
Ardalan Vahidi
Keyword(s):  
Monte Carlo ◽  
Real Time ◽  
Trajectory Planning ◽  
Hierarchy Theory ◽  
Lane Change ◽  
Tree Search ◽  
Behavior Prediction ◽  
Highway Traffic ◽  
Monte Carlo Tree Search ◽  
Cognitive Hierarchy

Abstract Predicting the states of the surrounding traffic is one of the major problems in automated driving. Maneuvers such as lane change, merge, and exit management could pose challenges in the absence of intervehicular communications and can benefit from driver behavior prediction. Predicting the motion of surrounding vehicles and trajectory planning need to be computationally efficient for real-time implementation. The main goal of this paper is to develop a fast algorithm that predicts the future states of the neighboring vehicles. The proposed workflow employs Monte Carlo Tree Search (MCTS) along with an on-policy learning technique for fast trajectory planning in multi-lane highway traffic scenarios. Also, for the inclusion of behavioral aspects, cognitive hierarchy and level-K game theories are utilized to predict the reaction and decision of the surrounding drivers. Simulation case studies demonstrate that our proposed approach is real-time implementable and can often avoid collision in difficult simulated confrontations.

scholarly journals Combinatorial Multi-armed Bandits for Real-Time Strategy Games

2017 ◽  
Vol 58 ◽  
pp. 665-702 ◽  
Author(s):  
Santiago Ontañón
Keyword(s):  
Real Time ◽  
Computer Games ◽  
Sampling Strategy ◽  
The Other ◽  
Tree Search ◽  
Game Tree ◽  
Monte Carlo Tree Search ◽  
Strategy Games ◽  
Branching Factor ◽  
Game Tree Search

Games with large branching factors pose a significant challenge for game tree search algorithms. In this paper, we address this problem with a sampling strategy for Monte Carlo Tree Search (MCTS) algorithms called "naive sampling", based on a variant of the Multi-armed Bandit problem called "Combinatorial Multi-armed Bandits" (CMAB). We analyze the theoretical properties of several variants of naive sampling, and empirically compare it against the other existing strategies in the literature for CMABs. We then evaluate these strategies in the context of real-time strategy (RTS) games, a genre of computer games characterized by their very large branching factors. Our results show that as the branching factor grows, naive sampling outperforms the other sampling strategies.

scholarly journals Multiobjective Monte Carlo Tree Search for Real-Time Games

2015 ◽  
Vol 7 (4) ◽  
pp. 347-360 ◽  
Author(s):  
Diego Perez ◽  
Sanaz Mostaghim ◽  
Spyridon Samothrakis ◽  
Simon M. Lucas
Keyword(s):  
Monte Carlo ◽  
Real Time ◽  
Tree Search ◽  
Monte Carlo Tree Search

scholarly journals Tackling Sparse Rewards in Real-Time Games with Statistical Forward Planning Methods

2019 ◽  
Vol 33 ◽  
pp. 1691-1698 ◽  
Author(s):  
Raluca D. Gaina ◽  
Simon M. Lucas ◽  
Diego Pérez-Liébana
Keyword(s):  
Monte Carlo ◽  
Evolutionary Algorithms ◽  
Real Time ◽  
Fitness Landscape ◽  
Tree Search ◽  
Rolling Horizon ◽  
Reward Systems ◽  
Monte Carlo Tree Search ◽  
Planning Methods ◽  
High Level

One of the issues general AI game players are required to deal with is the different reward systems in the variety of games they are expected to be able to play at a high level. Some games may present plentiful rewards which the agents can use to guide their search for the best solution, whereas others feature sparse reward landscapes that provide little information to the agents. The work presented in this paper focuses on the latter case, which most agents struggle with. Thus, modifications are proposed for two algorithms, Monte Carlo Tree Search and Rolling Horizon Evolutionary Algorithms, aiming at improving performance in this type of games while maintaining overall win rate across those where rewards are plentiful. Results show that longer rollouts and individual lengths, either fixed or responsive to changes in fitness landscape features, lead to a boost of performance in the games during testing without being detrimental to non-sparse reward scenarios.

Monte Carlo Tree Search-Based Mixed Traffic Flow Control Algorithm for Arterial Intersections

2020 ◽  
Vol 2674 (8) ◽  
pp. 167-178
Author(s):  
Yanqiu Cheng ◽  
Xianbiao Hu ◽  
Qing Tang ◽  
Hongsheng Qi ◽  
Hong Yang
Keyword(s):  
Monte Carlo ◽  
Real Time ◽  
Traffic Flow ◽  
Tree Search ◽  
Mixed Traffic ◽  
Solution Quality ◽  
Monte Carlo Tree Search ◽  
Mixed Traffic Flow ◽  
Model Free ◽  
Tree Expansion

A model-free approach is presented, based on the Monte Carlo tree search (MCTS) algorithm, for the control of mixed traffic flow of human-driven vehicles (HDV) and connected and autonomous vehicles (CAV), named MCTS-MTF, on a one-lane roadway with signalized intersection control. Previous research has often simplified the problem with certain assumptions to reduce computational burden, such as dividing a vehicle trajectory into several segments with constant speed or linear acceleration/deceleration, which was rather unrealistic. This study departs from the existing research in that minimum constraints on CAV trajectory control were required, as long as the basic rules such as safety considerations and vehicular performance limits were followed. Modeling efforts were made to improve the algorithm solution quality and the run time efficiency over the naïve MCTS algorithm. This was achieved by an exploration-exploitation balance calibration module, and a tree expansion determination module to expand the tree more effectively along the desired direction. Results of a case study found that the proposed algorithm was able to achieve a travel time saving of 3.5% and a fuel consumption saving of 6.5%. It was also demonstrated to run at eight times the speed of a naïve MCTS model, suggesting a promising potential for real-time or near real-time applications.

Sign in / Sign up

Export Citation Format

Share Document