scholarly journals Co-Evolution of Predator-Prey Ecosystems by Reinforcement Learning Agents

Entropy ◽  
2021 ◽  
Vol 23 (4) ◽  
pp. 461
Author(s):  
Jeongho Park ◽  
Juwon Lee ◽  
Taehwan Kim ◽  
Inkyung Ahn ◽  
Jooyoung Park
Keyword(s):  
Reinforcement Learning ◽  
Expected Returns ◽  
Dynamic Nature ◽  
Predator Prey ◽  
Learning Agents ◽  
Complex Interactions ◽  
Multi Agent ◽  
Simulation Results ◽  
Multiple Species ◽  
And Robotics

The problem of finding adequate population models in ecology is important for understanding essential aspects of their dynamic nature. Since analyzing and accurately predicting the intelligent adaptation of multiple species is difficult due to their complex interactions, the study of population dynamics still remains a challenging task in computational biology. In this paper, we use a modern deep reinforcement learning (RL) approach to explore a new avenue for understanding predator-prey ecosystems. Recently, reinforcement learning methods have achieved impressive results in areas, such as games and robotics. RL agents generally focus on building strategies for taking actions in an environment in order to maximize their expected returns. Here we frame the co-evolution of predators and preys in an ecosystem as allowing agents to learn and evolve toward better ones in a manner appropriate for multi-agent reinforcement learning. Recent significant advancements in reinforcement learning allow for new perspectives on these types of ecological issues. Our simulation results show that throughout the scenarios with RL agents, predators can achieve a reasonable level of sustainability, along with their preys.

scholarly journals Arena: A General Evaluation Platform and Building Toolkit for Multi-Agent Intelligence

2020 ◽  
Vol 34 (05) ◽  
pp. 7253-7260 ◽  
Author(s):  
Yuhang Song ◽  
Andrzej Wojcicki ◽  
Thomas Lukasiewicz ◽  
Jianyi Wang ◽  
Abi Aryan ◽  
...  
Keyword(s):  
Reinforcement Learning ◽  
State Of The Art ◽  
Research Community ◽  
Learning Agents ◽  
General Evaluation ◽  
Agent Learning ◽  
Multi Agent ◽  
Agent Intelligence ◽  
Training Schemes ◽  
Evaluation Platform

Learning agents that are not only capable of taking tests, but also innovating is becoming a hot topic in AI. One of the most promising paths towards this vision is multi-agent learning, where agents act as the environment for each other, and improving each agent means proposing new problems for others. However, existing evaluation platforms are either not compatible with multi-agent settings, or limited to a specific game. That is, there is not yet a general evaluation platform for research on multi-agent intelligence. To this end, we introduce Arena, a general evaluation platform for multi-agent intelligence with 35 games of diverse logics and representations. Furthermore, multi-agent intelligence is still at the stage where many problems remain unexplored. Therefore, we provide a building toolkit for researchers to easily invent and build novel multi-agent problems from the provided game set based on a GUI-configurable social tree and five basic multi-agent reward schemes. Finally, we provide Python implementations of five state-of-the-art deep multi-agent reinforcement learning baselines. Along with the baseline implementations, we release a set of 100 best agents/teams that we can train with different training schemes for each game, as the base for evaluating agents with population performance. As such, the research community can perform comparisons under a stable and uniform standard. All the implementations and accompanied tutorials have been open-sourced for the community at https://sites.google.com/view/arena-unity/.

Combat Robot Strategy Adaptation Using Multiple Learning Agents

2012 ◽  
Author(s):  
Thomas Recchia ◽  
Jae Chung ◽  
Kishore Pochiraju
Keyword(s):  
Reinforcement Learning ◽  
Robotic Systems ◽  
Multi Agent System ◽  
Learning Agents ◽  
Loosely Coupled ◽  
Reward Function ◽  
Strategy Adaptation ◽  
Agent Learning ◽  
Multi Agent ◽  
Reward Functions

As robotic systems become more prevalent, it is highly desirable for them to be able to operate in highly dynamic environments. A common approach is to use reinforcement learning to allow an agent controlling the robot to learn and adapt its behavior based on a reward function. This paper presents a novel multi-agent system that cooperates to control a single robot battle tank in a melee battle scenario, with no prior knowledge of its opponents’ strategies. The agents learn through reinforcement learning, and are loosely coupled by their reward functions. Each agent controls a different aspect of the robot’s behavior. In addition, the problem of delayed reward is addressed through a time-averaged reward applied to several sequential actions at once. This system was evaluated in a simulated melee combat scenario and was shown to learn to improve its performance over time. This was accomplished by each agent learning to pick specific battle strategies for each different opponent it faced.

Designing Internal Reward of Reinforcement Learning Agents in Multi-Step Dilemma Problem

2013 ◽  
Vol 17 (6) ◽  
pp. 926-931 ◽  
Author(s):  
Yoshihiro Ichikawa ◽  
◽  
Keiki Takadama
Keyword(s):  
Reinforcement Learning ◽  
Local Convergence ◽  
Average Reward ◽  
Q Value ◽  
Learning Agents ◽  
Learning Agent ◽  
Simulation Results ◽  
External Reward

This paper proposes the reinforcement learning agent that estimates internal rewards using external rewards in order to avoid conflict in multi-step dilemma problem. Intensive simulation results have revealed that the agent succeeds in avoiding local convergence and obtains a behavior policy for reaching a higher reward by updating the Q-value using the value that is subtracted the average reward from an external reward.

Building autonomic systems using collaborative reinforcement learning

2006 ◽  
Vol 21 (3) ◽  
pp. 231-238 ◽  
Author(s):  
JIM DOWLING ◽  
RAYMOND CUNNINGHAM ◽  
EOIN CURRAN ◽  
VINNY CAHILL
Keyword(s):  
Reinforcement Learning ◽  
Ad Hoc ◽  
Optimization Problems ◽  
System Optimization ◽  
Multi Agent Systems ◽  
Learning Agents ◽  
Coordination Model ◽  
Routing Performance ◽  
Multi Agent ◽  
Unpredictable Environment

This paper presents Collaborative Reinforcement Learning (CRL), a coordination model for online system optimization in decentralized multi-agent systems. In CRL system optimization problems are represented as a set of discrete optimization problems, each of whose solution cost is minimized by model-based reinforcement learning agents collaborating on their solution. CRL systems can be built to provide autonomic behaviours such as optimizing system performance in an unpredictable environment and adaptation to partial failures. We evaluate CRL using an ad hoc routing protocol that optimizes system routing performance in an unpredictable network environment.

scholarly journals Research on Tensor-Based Cooperative and Competitive in Multi-Agent Reinforcement Learning

2020 ◽  
Vol 4 (6) ◽  
Author(s):  
Tsega Weldu Araya ◽  
Md Rashed Ibn Nawab ◽  
A. P. Yuan Ling
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Data Representation ◽  
Training Data ◽  
Two Dimensional ◽  
Multiple Agents ◽  
Learning Agents ◽  
Dimensional Array ◽  
Multi Agent ◽  
Agent Cooperation

As technology overgrows, the assortment of information and the density of work becomes demanding to manage. To resolve the density of employment and human labor, machine-learning (ML) technology developed. Reinforcement learning (RL) is the recent advancement of ML studies. Multi-agent reinforcement learning (MARL) is useful to train multiple agents in the surrounding environment. The previous research studies focused on two-agent cooperation. Their data representation was held in a two-dimensional array, which is called a matrix. The limitation of this two-dimensional array appears as the training data of agents increases. The growth in the training data of agents creates storage drawbacks and data redundancy. Our first aim in this research is to improve an algorithm that can represent MARL training in tensor. In MARL, multiple agents are work together to achieve joint work. To share the training records and data of numerous agents, we need to collect the previous cumulative experience of agents in tensor. Secondly, we will discover the agent's cooperation and competition, with local and global goals of agents in MARL. Local goals are the cooperation of agents in a group or team where we use the training model as a student and teacher agent. The global goal is the competition between two contrary teams to acquire the reward. All learning agents have their Q table for storing the individual agent's training data in an environment. The growth in the number of learning agents, their training experience in Q tables, and the requirement for representing multiple data become the most challenging issue. We introduce tensor to store various data to resolve the challenges for data representation in multiple agent associations. Tensor is expressed as the three-dimensional array, although it is an N-way array, which is useful for representing and accessing numerous data. Finally, we will implement an algorithm for learning three cooperative agents against the opposed team using a tensor-based framework in the Q learning algorithm. We will provide an algorithm that can store the training records and data of multiple agents. Tensor advances to get a small storage size than the matrix for the training records of agents. Although three agent cooperation benefits to having maximum optimal reward.

scholarly journals Deep Reinforcement Learning Algorithms for Multiple Arc-Welding Robots

2021 ◽  
Vol 2 ◽  
Author(s):  
Lei-Xin Xu ◽  
Yang-Yang Chen
Keyword(s):  
Reinforcement Learning ◽  
Arc Welding ◽  
Distributed Simulation ◽  
Local Information ◽  
Time Varying ◽  
Robot Systems ◽  
Policy Gradient ◽  
Distributed Execution ◽  
Multi Agent ◽  
Simulation Results

The applications of the deep reinforcement learning method to achieve the arcs welding by multi-robot systems are presented, where the states and the actions of each robot are continuous and obstacles are considered in the welding environment. In order to adapt to the time-varying welding task and local information available to each robot in the welding environment, the so-called multi-agent deep deterministic policy gradient (MADDPG) algorithm is designed with a new set of rewards. Based on the idea of the distributed execution and centralized training, the proposed MADDPG algorithm is distributed. Simulation results demonstrate the effectiveness of the proposed method.

scholarly journals Alternation Measures for the Evaluation of Selfish Agents’ Turn-Taking

2021 ◽  
Author(s):  
Nikolaos Al. Papadopoulos ◽  
Marti Sanchez-Fibla
Keyword(s):  
Reinforcement Learning ◽  
State Of The Art ◽  
Social Phenomena ◽  
Learning Agents ◽  
Fairness And Efficiency ◽  
Turn Taking ◽  
Multi Agent ◽  
Series Of Experiments ◽  
Selfish Agents ◽  
Efficient Distribution

Multi-Agent Reinforcement Learning reductionist simulations can provide a spectrum of opportunities towards the modeling and understanding of complex social phenomena such as common-pool appropriation. In this paper, a multiplayer variant of Battle-of-the-Exes is suggested as appropriate for experimentation regarding fair and efficient coordination and turn-taking among selfish agents. Going beyond literature’s fairness and efficiency, a novel measure is proposed for turn-taking coordination evaluation, robust to the number of agents and episodes of a system. Six variants of this measure are defined, entitled Alternation Measures or ALT. ALT measures were found sufficient to capture the desired properties (alternation, fair and efficient distribution) in comparison to state-of-the-art measures, thus they were benchmarked and tested through a series of experiments with Reinforcement Learning agents, aspiring to contribute novel tools for a deeper understanding of emergent social outcomes.

Multi-Agent Reinforcement Learning for Value Co-Creation of Collaborative Transportation Management (CTM)

2017 ◽  
Vol 10 (3) ◽  
pp. 84-95 ◽  
Author(s):  
Liane Okdinawati ◽  
Togar M. Simatupang ◽  
Yos Sunitiyoso
Keyword(s):  
Reinforcement Learning ◽  
Business Process ◽  
Transportation Cost ◽  
Learning Processes ◽  
Transportation Management ◽  
Model Free ◽  
Multiple Case ◽  
Multi Agent ◽  
Multiple Case Studies ◽  
Simulation Results

Collaborative Transportation Management (CTM) is a collaboration model in transportation area. The use of CTM in today's business process is to create efficiency in transportation planning and execution processes. However, previous research paid little attention to demonstrate the ability for all agents in CTM to co-create value in services. The purpose of this paper is to increase the understanding of value co-creation in CTM area and learning processes in real systems based on value co-creation of CTM. Multiple case studies were used to analyze the value that was perceived by all agents in CTM in each collaboration stage and provided empirical evidence on the interactions among agents. Model-free reinforcement learning was used to predict how CTM could reduce transportation cost, increase visibility, and improve agility. The simulation results show that the input, feedback, and the experience of the agents are used to structure the collaboration processes and determine the strategies.

Sign in / Sign up

Export Citation Format

Share Document