Cooperative Behavior by Multi-agent Reinforcement Learning with Abstractive Communication

2019 ◽  
Author(s):  
Jin Tanda ◽  
Ahmed Moustafa ◽  
Takayuki Ito
Keyword(s):  
Reinforcement Learning ◽  
Cooperative Behavior ◽  
Multi Agent

scholarly journals Two-stage training algorithm for AI robot soccer

2021 ◽  
Vol 7 ◽  
pp. e718
Author(s):  
Taeyoung Kim ◽  
Luiz Felipe Vecchietti ◽  
Kyujin Choi ◽  
Sanem Sariel ◽  
Dongsoo Har
Keyword(s):  
Reinforcement Learning ◽  
Heterogeneous Agents ◽  
Cooperative Behavior ◽  
Simulation Software ◽  
The Other ◽  
Learning Performance ◽  
Time Step ◽  
Two Stage ◽  
Robot Soccer ◽  
Multi Agent

In multi-agent reinforcement learning, the cooperative learning behavior of agents is very important. In the field of heterogeneous multi-agent reinforcement learning, cooperative behavior among different types of agents in a group is pursued. Learning a joint-action set during centralized training is an attractive way to obtain such cooperative behavior; however, this method brings limited learning performance with heterogeneous agents. To improve the learning performance of heterogeneous agents during centralized training, two-stage heterogeneous centralized training which allows the training of multiple roles of heterogeneous agents is proposed. During training, two training processes are conducted in a series. One of the two stages is to attempt training each agent according to its role, aiming at the maximization of individual role rewards. The other is for training the agents as a whole to make them learn cooperative behaviors while attempting to maximize shared collective rewards, e.g., team rewards. Because these two training processes are conducted in a series in every time step, agents can learn how to maximize role rewards and team rewards simultaneously. The proposed method is applied to 5 versus 5 AI robot soccer for validation. The experiments are performed in a robot soccer environment using Webots robot simulation software. Simulation results show that the proposed method can train the robots of the robot soccer team effectively, achieving higher role rewards and higher team rewards as compared to other three approaches that can be used to solve problems of training cooperative multi-agent. Quantitatively, a team trained by the proposed method improves the score concede rate by 5% to 30% when compared to teams trained with the other approaches in matches against evaluation teams.

scholarly journals Adaptation to Other Agent’s Behavior Using Meta-Strategy Learning by Collision Avoidance Simulation

2021 ◽  
Vol 11 (4) ◽  
pp. 1786
Author(s):  
Kensuke Miyamoto ◽  
Norifumi Watanabe ◽  
Yoshiyasu Takefuji
Keyword(s):  
Reinforcement Learning ◽  
Collision Avoidance ◽  
Cooperative Behavior ◽  
Behavioral Strategy ◽  
Active Strategy ◽  
Cooperative Tasks ◽  
Behavioral Choices ◽  
Multi Agent ◽  
High Degree ◽  
Passive Strategy

In human’s cooperative behavior, there are two strategies: a passive behavioral strategy based on others’ behaviors and an active behavioral strategy based on the objective-first. However, it is not clear how to acquire a meta-strategy to switch those strategies. The purpose of the proposed study is to create agents with the meta-strategy and to enable complex behavioral choices with a high degree of coordination. In this study, we have experimented by using multi-agent collision avoidance simulations as an example of cooperative tasks. In the experiments, we have used reinforcement learning to obtain an active strategy and a passive strategy by rewarding the interaction with agents facing each other. Furthermore, we have examined and verified the meta-strategy in situations with opponent’s strategy switched.

Comparison Between Reinforcement Learning Methods with Different Goal Selections in Multi-Agent Cooperation

2017 ◽  
Vol 21 (5) ◽  
pp. 917-929 ◽  
Author(s):  
Fumito Uwano ◽  
◽  
Keiki Takadama
Keyword(s):  
Reinforcement Learning ◽  
Learning Process ◽  
Cooperative Behavior ◽  
Learning Methods ◽  
Q Learning ◽  
Designed Experiments ◽  
Multi Agent ◽  
Agent Cooperation ◽  
Maze Problem

This study discusses important factors for zero communication, multi-agent cooperation by comparing different modified reinforcement learning methods. The two learning methods used for comparison were assigned different goal selections for multi-agent cooperation tasks. The first method is called Profit Minimizing Reinforcement Learning (PMRL); it forces agents to learn how to reach the farthest goal, and then the agent closest to the goal is directed to the goal. The second method is called Yielding Action Reinforcement Learning (YARL); it forces agents to learn through a Q-learning process, and if the agents have a conflict, the agent that is closest to the goal learns to reach the next closest goal. To compare the two methods, we designed experiments by adjusting the following maze factors: (1) the location of the start point and goal; (2) the number of agents; and (3) the size of maze. The intensive simulations performed on the maze problem for the agent cooperation task revealed that the two methods successfully enabled the agents to exhibit cooperative behavior, even if the size of the maze and the number of agents change. The PMRL mechanism always enables the agents to learn cooperative behavior, whereas the YARL mechanism makes the agents learn cooperative behavior over a small number of learning iterations. In zero communication, multi-agent cooperation, it is important that only agents that have a conflict cooperate with each other.

Multi-agent deep reinforcement learning with type-based hierarchical group communication

2021 ◽  
Author(s):  
Hao Jiang ◽  
Dianxi Shi ◽  
Chao Xue ◽  
Yajie Wang ◽  
Gongju Wang ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
Group Communication ◽  
Multi Agent ◽  
Hierarchical Group
Sign in / Sign up

Export Citation Format

Share Document