Reinforcement Learning Hierarchical Neuro-Fuzzy Politree Model for Control of Autonomous Agents

2005 ◽  
Author(s):  
K. Figueiredo ◽  
M. Vellasco ◽  
M. Pacheco ◽  
F. Souza
Keyword(s):  
Reinforcement Learning ◽  
Autonomous Agents ◽  
Neuro Fuzzy

scholarly journals A strategy learning model for autonomous agents based on classification

2015 ◽  
Vol 25 (3) ◽  
pp. 471-482 ◽  
Author(s):  
Bartłomiej Śnieżyński
Keyword(s):  
Reinforcement Learning ◽  
Supervised Learning ◽  
Learning Process ◽  
Autonomous Agents ◽  
Good Alternative ◽  
Learning Model ◽  
Learning Method ◽  
Complex Environments ◽  
Agent Based ◽  
Proposed Model

AbstractIn this paper we propose a strategy learning model for autonomous agents based on classification. In the literature, the most commonly used learning method in agent-based systems is reinforcement learning. In our opinion, classification can be considered a good alternative. This type of supervised learning can be used to generate a classifier that allows the agent to choose an appropriate action for execution. Experimental results show that this model can be successfully applied for strategy generation even if rewards are delayed. We compare the efficiency of the proposed model and reinforcement learning using the farmer-pest domain and configurations of various complexity. In complex environments, supervised learning can improve the performance of agents much faster that reinforcement learning. If an appropriate knowledge representation is used, the learned knowledge may be analyzed by humans, which allows tracking the learning process

scholarly journals What Can You Do with a Rock? Affordance Extraction via Word Embeddings

2017 ◽  
Author(s):  
Nancy Fulda ◽  
Daniel Ricks ◽  
Ben Murdoch ◽  
David Wingate
Keyword(s):  
Reinforcement Learning ◽  
Computational Complexity ◽  
Linear Algebra ◽  
Autonomous Agents ◽  
Common Knowledge ◽  
Search Space ◽  
Word Embeddings ◽  
Knowledge Database ◽  
Learning Agent ◽  
Action Spaces

Autonomous agents must often detect affordances: the set of behaviors enabled by a situation. Affordance extraction is particularly helpful in domains with large action spaces, allowing the agent to prune its search space by avoiding futile behaviors. This paper presents a method for affordance extraction via word embeddings trained on a tagged Wikipedia corpus. The resulting word vectors are treated as a common knowledge database which can be queried using linear algebra. We apply this method to a reinforcement learning agent in a text-only environment and show that affordance-based action selection improves performance in most cases. Our method increases the computational complexity of each learning step but significantly reduces the total number of steps needed. In addition, the agent's action selections begin to resemble those a human would choose.

Evolutionary Control of Helicopter Hovering Based on Genetic Programming

2013 ◽  
pp. 1-19
Author(s):  
Dimitris C. Dracopoulos ◽  
Dimitrios Effraimidis
Keyword(s):  
Reinforcement Learning ◽  
Genetic Programming ◽  
Control Problems ◽  
Complex Control ◽  
Fuzzy Methods ◽  
Neuro Fuzzy ◽  
Complex Control System ◽  
And Control ◽  
Intelligent Controllers ◽  
Hovering Control

Computational intelligence techniques such as neural networks, fuzzy logic, and hybrid neuroevolutionary and neuro-fuzzy methods have been successfully applied to complex control problems in the last two decades. Genetic programming, a field under the umbrella of evolutionary computation, has not been applied to a sufficiently large number of challenging and difficult control problems, in order to check its viability as a general methodology to such problems. Helicopter hovering control is considered a challenging control problem in the literature and has been included in the set of benchmarks of recent reinforcement learning competitions for deriving new intelligent controllers. This chapter shows how genetic programming can be applied for the derivation of controllers in this nonlinear, high dimensional, complex control system. The evolved controllers are compared with a neuroevolutionary approach that won the first position in the 2008 helicopter hovering reinforcement learning competition. The two approaches perform similarly (and in some cases GP performs better than the winner of the competition), even in the case where unknown wind is added to the dynamic system and control is based on structures evolved previously, that is, the evolved controllers have good generalization capability.

Hierarchical Neuro-Fuzzy Systems Part II

2011 ◽  
pp. 817-824
Author(s):  
Marley Vellasco ◽  
Marco Pacheco ◽  
Karla Figueiredo ◽  
Flavio Souza
Keyword(s):  
Reinforcement Learning ◽  
State Space ◽  
Learning Process ◽  
Fuzzy Systems ◽  
Space Partitioning ◽  
Learning Methods ◽  
New Class ◽  
Neuro Fuzzy ◽  
Binary Space Partitioning ◽  
Priori Information

This paper describes a new class of neuro-fuzzy models, called Reinforcement Learning Hierarchical Neuro- Fuzzy Systems (RL-HNF). These models employ the BSP (Binary Space Partitioning) and Politree partitioning of the input space [Chrysanthou,1992] and have been developed in order to bypass traditional drawbacks of neuro-fuzzy systems: the reduced number of allowed inputs and the poor capacity to create their own structure and rules (ANFIS [Jang,1997], NEFCLASS [Kruse,1995] and FSOM [Vuorimaa,1994]). These new models, named Reinforcement Learning Hierarchical Neuro-Fuzzy BSP (RL-HNFB) and Reinforcement Learning Hierarchical Neuro-Fuzzy Politree (RL-HNFP), descend from the original HNFB that uses Binary Space Partitioning (see Hierarchical Neuro-Fuzzy Systems Part I). By using hierarchical partitioning, together with the Reinforcement Learning (RL) methodology, a new class of Neuro-Fuzzy Systems (SNF) was obtained, which executes, in addition to automatically learning its structure, the autonomous learning of the actions to be taken by an agent, dismissing a priori information (number of rules, fuzzy rules and sets) relative to the learning process. These characteristics represent an important differential when compared with existing intelligent agents learning systems, because in applications involving continuous environments and/or environments considered to be highly dimensional, the use of traditional Reinforcement Learning methods based on lookup tables (a table that stores value functions for a small or discrete state space) is no longer possible, since the state space becomes too large. This second part of hierarchical neuro-fuzzy systems focus on the use of reinforcement learning process. The first part presented HNFB models based on supervised learning methods. The RL-HNFB and RL-HNFP models were evaluated in a benchmark control application and a simulated Khepera robot environment with multiple obstacles.

Sign in / Sign up

Export Citation Format

Share Document