scholarly journals State Space Construction Method with Self-organizing Map in a Reinforcement Learning System Based on Profit Sharing

2008 ◽  
Vol 20 (3) ◽  
pp. 369-378
Author(s):  
Fumiaki SAITOH ◽  
Osamu HASEGAWA
Keyword(s):  
Reinforcement Learning ◽  
State Space ◽  
Profit Sharing ◽  
Construction Method ◽  
Learning System ◽  
Self Organizing Map ◽  
Space Construction ◽  
Self Organizing

scholarly journals Parameterless-Growing-SOM and Its Application to a Voice Instruction Learning System

2010 ◽  
Vol 2010 ◽  
pp. 1-9 ◽  
Author(s):  
Takashi Kuremoto ◽  
Takahito Komoto ◽  
Kunikazu Kobayashi ◽  
Masanao Obayashi
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Learning System ◽  
Automatic Tuning ◽  
Self Organizing Map ◽  
Multiple Languages ◽  
Voice Instruction ◽  
Japanese English ◽  
Neighborhood Preservation ◽  
Self Organizing

An improved self-organizing map (SOM), parameterless-growing-SOM (PL-G-SOM), is proposed in this paper. To overcome problems existed in traditional SOM (Kohonen, 1982), kinds of structure-growing-SOMs or parameter-adjusting-SOMs have been invented and usually separately. Here, we combine the idea of growing SOMs (Bauer and Villmann, 1997; Dittenbach et al. 2000) and a parameterless SOM (Berglund and Sitte, 2006) together to be a novel SOM named PL-G-SOM to realize additional learning, optimal neighborhood preservation, and automatic tuning of parameters. The improved SOM is applied to construct a voice instruction learning system for partner robots adopting a simple reinforcement learning algorithm. User's instructions of voices are classified by the PL-G-SOM at first, then robots choose an expected action according to a stochastic policy. The policy is adjusted by the reward/punishment given by the user of the robot. A feeling map is also designed to express learning degrees of voice instructions. Learning and additional learning experiments used instructions in multiple languages including Japanese, English, Chinese, and Malaysian confirmed the effectiveness of our proposed system.

scholarly journals Mental state space visualization for interactive modeling of personalized BCI control strategies

2019 ◽  
Author(s):  
Ilya Kuzovkin ◽  
Konstantin Tretyakov ◽  
Andero Uusberg ◽  
Raul Vicente
Keyword(s):  
State Space ◽  
Mental State ◽  
Control Strategies ◽  
Muscular Activity ◽  
Mental States ◽  
Learning System ◽  
Self Organizing Map ◽  
Preliminary Validation ◽  
Som Algorithm ◽  
Mental Actions

AbstractObjectiveNumerous studies in the area of BCI are focused on the search for a better experimental paradigm – a set of mental actions that a user can evoke consistently and a machine can discriminate reliably. Examples of such mental activities are motor imagery, mental computations, etc. We propose a technique that instead allows the user to try different mental actions in the search for the ones that will work best.ApproachThe system is based on a modification of the self-organizing map (SOM) algorithm and enables interactive communication between the user and the learning system through a visualization of user’s mental state space. During the interaction with the system the user converges on the paradigm that is most efficient and intuitive for that particular user.Main resultsResults of the two experiments, one allowing muscular activity, another permitting mental activity only, demonstrate soundness of the proposed method and offer preliminary validation of the performance improvement over the traditional closed-loop feedback approach.SignificanceThe proposed method allows a user to visually explore their mental state space in real time, opening new opportunities for scientific inquiry. The application of this method to the area of brain-computer interfaces enables more efficient search for the mental states that will allow a user to reliably control a BCI system.

scholarly journals Self-organizing maps for storage and transfer of knowledge in reinforcement learning

2018 ◽  
Vol 27 (2) ◽  
pp. 111-126 ◽  
Author(s):  
Thommen George Karimpanal ◽  
Roland Bouffanais
Keyword(s):  
Reinforcement Learning ◽  
Self Organizing Map ◽  
Value Functions ◽  
Transfer Of Knowledge ◽  
Network Growth ◽  
Self Organizing Maps ◽  
Task Knowledge ◽  
Novel Approach ◽  
Learning Agent ◽  
Self Organizing

The idea of reusing or transferring information from previously learned tasks (source tasks) for the learning of new tasks (target tasks) has the potential to significantly improve the sample efficiency of a reinforcement learning agent. In this work, we describe a novel approach for reusing previously acquired knowledge by using it to guide the exploration of an agent while it learns new tasks. In order to do so, we employ a variant of the growing self-organizing map algorithm, which is trained using a measure of similarity that is defined directly in the space of the vectorized representations of the value functions. In addition to enabling transfer across tasks, the resulting map is simultaneously used to enable the efficient storage of previously acquired task knowledge in an adaptive and scalable manner. We empirically validate our approach in a simulated navigation environment and also demonstrate its utility through simple experiments using a mobile micro-robotics platform. In addition, we demonstrate the scalability of this approach and analytically examine its relation to the proposed network growth mechanism. Furthermore, we briefly discuss some of the possible improvements and extensions to this approach, as well as its relevance to real-world scenarios in the context of continual learning.

Sign in / Sign up

Export Citation Format

Share Document