scholarly journals Training and inferring neural network function with multi-agent reinforcement learning

2019 ◽  
Author(s):  
Matthew Chalk ◽  
Gasper Tkacik ◽  
Olivier Marre
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Recurrent Network ◽  
Neural Systems ◽  
Neural Recordings ◽  
Reward Function ◽  
Network Function ◽  
Structure Changes ◽  
Theoretical Predictions ◽  
Multi Agent

AbstractA central goal in systems neuroscience is to understand the functions performed by neural circuits. Previous top-down models addressed this question by comparing the behaviour of an ideal model circuit, optimised to perform a given function, with neural recordings. However, this requires guessing in advance what function is being performed, which may not be possible for many neural systems. To address this, we propose a new framework for optimising a recurrent network using multi-agent reinforcement learning (RL). In this framework, a reward function quantifies how desirable each state of the network is for performing a given function. Each neuron is treated as an ‘agent’, which optimises its responses so as to drive the network towards rewarded states. Three applications follow from this. First, one can use multi-agent RL algorithms to optimise a recurrent neural network to perform diverse functions (e.g. efficient sensory coding or motor control). Second, one could use inverse RL to infer the function of a recorded neural network from data. Third, the theory predicts how neural networks should adapt their dynamics to maintain the same function when the external environment or network structure changes. This could lead to theoretical predictions about how neural network dynamics adapt to deal with cell death and/or varying sensory stimulus statistics.

scholarly journals Inferring the function performed by a recurrent neural network

PLoS ONE ◽  
2021 ◽  
Vol 16 (4) ◽  
pp. e0248940
Author(s):  
Matthew Chalk ◽  
Gasper Tkacik ◽  
Olivier Marre
Keyword(s):  
Neural Network ◽  
Neural Systems ◽  
Inverse Reinforcement Learning ◽  
Neural Recordings ◽  
Reward Function ◽  
Structure Changes ◽  
The Neural Network ◽  
Theoretical Predictions ◽  
Neural Network Dynamics ◽  
Model Circuit

A central goal in systems neuroscience is to understand the functions performed by neural circuits. Previous top-down models addressed this question by comparing the behaviour of an ideal model circuit, optimised to perform a given function, with neural recordings. However, this requires guessing in advance what function is being performed, which may not be possible for many neural systems. To address this, we propose an inverse reinforcement learning (RL) framework for inferring the function performed by a neural network from data. We assume that the responses of each neuron in a network are optimised so as to drive the network towards ‘rewarded’ states, that are desirable for performing a given function. We then show how one can use inverse RL to infer the reward function optimised by the network from observing its responses. This inferred reward function can be used to predict how the neural network should adapt its dynamics to perform the same function when the external environment or network structure changes. This could lead to theoretical predictions about how neural network dynamics adapt to deal with cell death and/or varying sensory stimulus statistics.

Combat Robot Strategy Adaptation Using Multiple Learning Agents

2012 ◽  
Author(s):  
Thomas Recchia ◽  
Jae Chung ◽  
Kishore Pochiraju
Keyword(s):  
Reinforcement Learning ◽  
Robotic Systems ◽  
Multi Agent System ◽  
Learning Agents ◽  
Loosely Coupled ◽  
Reward Function ◽  
Strategy Adaptation ◽  
Agent Learning ◽  
Multi Agent ◽  
Reward Functions

As robotic systems become more prevalent, it is highly desirable for them to be able to operate in highly dynamic environments. A common approach is to use reinforcement learning to allow an agent controlling the robot to learn and adapt its behavior based on a reward function. This paper presents a novel multi-agent system that cooperates to control a single robot battle tank in a melee battle scenario, with no prior knowledge of its opponents’ strategies. The agents learn through reinforcement learning, and are loosely coupled by their reward functions. Each agent controls a different aspect of the robot’s behavior. In addition, the problem of delayed reward is addressed through a time-averaged reward applied to several sequential actions at once. This system was evaluated in a simulated melee combat scenario and was shown to learn to improve its performance over time. This was accomplished by each agent learning to pick specific battle strategies for each different opponent it faced.

Simulation of football sport PID controller based on BP neural network

2020 ◽  
pp. 1-13
Author(s):  
L.V. Qiangguo
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Bp Neural Network ◽  
Pid Controller ◽  
Single Agent ◽  
Control Effect ◽  
Learning Problem ◽  
Learning Space ◽  
Multi Agent ◽  
Multi Agents

Multi-agent reinforcement learning in football simulation can be extended by single-agent reinforcement learning. However, compared with single agents, the learning space of multi-agents will increase dramatically with the increase in the number of agents, so the learning difficulty will also increase. Based on BP neural network as the model structure foundation, this research combines PID controller to control the process of model operation. In order to improve the calculation accuracy to improve the control effect, the prediction output is obtained through the prediction model instead of the actual measured value. In addition, with the football robot as the object, this research studies the multi-agent reinforcement learning problem and its application in the football robot. The content includes single-agent reinforcement learning, multi-agent system reinforcement learning, and ball hunting, role assignment, and action selection in football robot decision strategies based on this. The simulation results show that the method proposed in this paper has certain effects.

Sign in / Sign up

Export Citation Format

Share Document