scholarly journals Hybrid Online and Offline Reinforcement Learning for Tibetan Jiu Chess

Complexity ◽  
2020 ◽  
Vol 2020 ◽  
pp. 1-11
Author(s):  
Xiali Li ◽  
Zhengyu Lv ◽  
Licheng Wu ◽  
Yue Zhao ◽  
Xiaona Xu
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Domain Knowledge ◽  
Deep Neural Network ◽  
Learning Strategy ◽  
Learning Algorithms ◽  
Hybrid State ◽  
Q Learning ◽  
State Action ◽  
Upper Confidence Bound

In this study, hybrid state-action-reward-state-action (SARSAλ) and Q-learning algorithms are applied to different stages of an upper confidence bound applied to tree search for Tibetan Jiu chess. Q-learning is also used to update all the nodes on the search path when each game ends. A learning strategy that uses SARSAλ and Q-learning algorithms combining domain knowledge for a feedback function for layout and battle stages is proposed. An improved deep neural network based on ResNet18 is used for self-play training. Experimental results show that hybrid online and offline reinforcement learning with a deep neural network can improve the game program’s learning efficiency and understanding ability for Tibetan Jiu chess.

scholarly journals Image Classification Using Reinforcement Learning

2020 ◽  
Vol 23 (6) ◽  
pp. 1172-1191
Author(s):  
Artem Aleksandrovich Elizarov ◽  
Evgenii Viktorovich Razinkov
Keyword(s):  
Neural Network ◽  
Artificial Intelligence ◽  
Machine Learning ◽  
Computer Vision ◽  
Reinforcement Learning ◽  
Image Classification ◽  
Deep Neural Network ◽  
Learning Algorithms ◽  
Further Development

Recently, such a direction of machine learning as reinforcement learning has been actively developing. As a consequence, attempts are being made to use reinforcement learning for solving computer vision problems, in particular for solving the problem of image classification. The tasks of computer vision are currently one of the most urgent tasks of artificial intelligence. The article proposes a method for image classification in the form of a deep neural network using reinforcement learning. The idea of ​​the developed method comes down to solving the problem of a contextual multi-armed bandit using various strategies for achieving a compromise between exploitation and research and reinforcement learning algorithms. Strategies such as -greedy, -softmax, -decay-softmax, and the UCB1 method, and reinforcement learning algorithms such as DQN, REINFORCE, and A2C are considered. The analysis of the influence of various parameters on the efficiency of the method is carried out, and options for further development of the method are proposed.

Introspective Q-learning and learning from demonstration

2019 ◽  
Vol 34 ◽  
Author(s):  
Mao Li ◽  
Tim Brys ◽  
Daniel Kudenko
Keyword(s):  
Reinforcement Learning ◽  
Potential Function ◽  
Domain Knowledge ◽  
Experimental Validation ◽  
Priority Queue ◽  
Learning From Demonstration ◽  
Q Learning ◽  
State Action ◽  
Reward Shaping ◽  
Speed Up

Abstract One challenge faced by reinforcement learning (RL) agents is that in many environments the reward signal is sparse, leading to slow improvement of the agent’s performance in early learning episodes. Potential-based reward shaping can help to resolve the aforementioned issue of sparse reward by incorporating an expert’s domain knowledge into the learning through a potential function. Past work on reinforcement learning from demonstration (RLfD) directly mapped (sub-optimal) human expert demonstration to a potential function, which can speed up RL. In this paper we propose an introspective RL agent that significantly further speeds up the learning. An introspective RL agent records its state–action decisions and experience during learning in a priority queue. Good quality decisions, according to a Monte Carlo estimation, will be kept in the queue, while poorer decisions will be rejected. The queue is then used as demonstration to speed up RL via reward shaping. A human expert’s demonstration can be used to initialize the priority queue before the learning process starts. Experimental validation in the 4-dimensional CartPole domain and the 27-dimensional Super Mario AI domain shows that our approach significantly outperforms non-introspective RL and state-of-the-art approaches in RLfD in both domains.

scholarly journals Fully distributed actor-critic architecture for multitask deep reinforcement learning

2021 ◽  
Vol 36 ◽  
Author(s):  
Sergio Valcarcel Macua ◽  
Ian Davies ◽  
Aleksi Tukiainen ◽  
Enrique Munoz de Cote
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Duality Theory ◽  
Deep Neural Network ◽  
Original Problem ◽  
Almost Sure Convergence ◽  
Continuous Control ◽  
Access Data ◽  
Central Station ◽  
Common Policy

Abstract We propose a fully distributed actor-critic architecture, named diffusion-distributed-actor-critic Diff-DAC, with application to multitask reinforcement learning (MRL). During the learning process, agents communicate their value and policy parameters to their neighbours, diffusing the information across a network of agents with no need for a central station. Each agent can only access data from its local task, but aims to learn a common policy that performs well for the whole set of tasks. The architecture is scalable, since the computational and communication cost per agent depends on the number of neighbours rather than the overall number of agents. We derive Diff-DAC from duality theory and provide novel insights into the actor-critic framework, showing that it is actually an instance of the dual-ascent method. We prove almost sure convergence of Diff-DAC to a common policy under general assumptions that hold even for deep neural network approximations. For more restrictive assumptions, we also prove that this common policy is a stationary point of an approximation of the original problem. Numerical results on multitask extensions of common continuous control benchmarks demonstrate that Diff-DAC stabilises learning and has a regularising effect that induces higher performance and better generalisation properties than previous architectures.

scholarly journals Optimising Performance for NB-IoT UE Devices through Data Driven Models

2021 ◽  
Vol 10 (1) ◽  
pp. 21
Author(s):  
Omar Nassef ◽  
Toktam Mahmoodi ◽  
Foivos Michelinakis ◽  
Kashif Mahmood ◽  
Ahmed Elmokashfi
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Gradient Descent ◽  
Deep Neural Network ◽  
Narrow Band ◽  
Learning Algorithm ◽  
Base Station ◽  
User Equipment ◽  
Data Driven ◽  
Superior Performance

This paper presents a data driven framework for performance optimisation of Narrow-Band IoT user equipment. The proposed framework is an edge micro-service that suggests one-time configurations to user equipment communicating with a base station. Suggested configurations are delivered from a Configuration Advocate, to improve energy consumption, delay, throughput or a combination of those metrics, depending on the user-end device and the application. Reinforcement learning utilising gradient descent and genetic algorithm is adopted synchronously with machine and deep learning algorithms to predict the environmental states and suggest an optimal configuration. The results highlight the adaptability of the Deep Neural Network in the prediction of intermediary environmental states, additionally the results present superior performance of the genetic reinforcement learning algorithm regarding its performance optimisation.

scholarly journals Reinforcement Learning using Convolutional Neural Network for Game Prediction

2020 ◽  
Vol 9 (8) ◽  
pp. 425-430
Keyword(s):  
Neural Network ◽  
Reinforcement Learning ◽  
Computer Games ◽  
Elevated Level ◽  
Q Learning ◽  
The Third ◽  
Level Information ◽  
Simple Neural Network ◽  
Fully Connected ◽  
Deep Learning Model

The paper presents a Deep learning model for playing computer games with elevated level information utilizing Reinforcement learning learning. The games are activity restricted (like snakes, catcher, air-bandit and so on.). The implementation is progressive in three parts. The first part deals with a simple neural network, the second one with Deep Q network and further to increase the accuracy and speed of the algorithm, the third part consists of a model consisting of convolution neural network for image processing and giving outputs from the fully connected layers so as to estimate the probability of an action being taken based on information extracted from inputs where we apply Q-learning to determine the best possible move. The results are further analysed and compared to provide an overview of the improvements in each methods.

Cloud Load Balancing and Reinforcement Learning

2018 ◽  
pp. 266-291
Author(s):  
Abdelghafour Harraz ◽  
Mostapha Zbakh
Keyword(s):  
Artificial Intelligence ◽  
Reinforcement Learning ◽  
Load Balancing ◽  
Decision Process ◽  
Cloud System ◽  
Human Intervention ◽  
Q Learning ◽  
State Action ◽  
Learning Techniques ◽  
Markov Decision

Artificial Intelligence allows to create engines that are able to explore, learn environments and therefore create policies that permit to control them in real time with no human intervention. It can be applied, through its Reinforcement Learning techniques component, using frameworks such as temporal differences, State-Action-Reward-State-Action (SARSA), Q Learning to name a few, to systems that are be perceived as a Markov Decision Process, this opens door in front of applying Reinforcement Learning to Cloud Load Balancing to be able to dispatch load dynamically to a given Cloud System. The authors will describe different techniques that can used to implement a Reinforcement Learning based engine in a cloud system.

scholarly journals Deep Reinforcement Learning by Balancing Offline Monte Carlo and Online Temporal Difference Use Based on Environment Experiences

Symmetry ◽  
2020 ◽  
Vol 12 (10) ◽  
pp. 1685 ◽  
Author(s):  
Chayoung Kim
Keyword(s):  
Monte Carlo ◽  
Reinforcement Learning ◽  
Real Time ◽  
Temporal Difference ◽  
Q Learning ◽  
State Action ◽  
Proposed Model ◽  
Reward Functions ◽  
And Performance ◽  
The Internet Of Things

Owing to the complexity involved in training an agent in a real-time environment, e.g., using the Internet of Things (IoT), reinforcement learning (RL) using a deep neural network, i.e., deep reinforcement learning (DRL) has been widely adopted on an online basis without prior knowledge and complicated reward functions. DRL can handle a symmetrical balance between bias and variance—this indicates that the RL agents are competently trained in real-world applications. The approach of the proposed model considers the combinations of basic RL algorithms with online and offline use based on the empirical balances of bias–variance. Therefore, we exploited the balance between the offline Monte Carlo (MC) technique and online temporal difference (TD) with on-policy (state-action–reward-state-action, Sarsa) and an off-policy (Q-learning) in terms of a DRL. The proposed balance of MC (offline) and TD (online) use, which is simple and applicable without a well-designed reward, is suitable for real-time online learning. We demonstrated that, for a simple control task, the balance between online and offline use without an on- and off-policy shows satisfactory results. However, in complex tasks, the results clearly indicate the effectiveness of the combined method in improving the convergence speed and performance in a deep Q-network.

scholarly journals Online Tuning of a PID Controller with a Fuzzy Reinforcement Learning MAS for Flow Rate Control of a Desalination Unit

Electronics ◽  
2019 ◽  
Vol 8 (2) ◽  
pp. 231 ◽  
Author(s):  
Panagiotis Kofinas ◽  
Anastasios I. Dounis
Keyword(s):  
Reinforcement Learning ◽  
Flow Rate ◽  
Pid Controller ◽  
Hybrid Control ◽  
Q Learning ◽  
State Action ◽  
Continuous State ◽  
Multi Agent ◽  
Flow Rate Control ◽  
Online Tuning

This paper proposes a hybrid Zeigler-Nichols (Z-N) fuzzy reinforcement learning MAS (Multi-Agent System) approach for online tuning of a Proportional Integral Derivative (PID) controller in order to control the flow rate of a desalination unit. The PID gains are set by the Z-N method and then are adapted online through the fuzzy Q-learning MAS. The fuzzy Q-learning is introduced in each agent in order to confront with the continuous state-action space. The global state of the MAS is defined by the value of the error and the derivative of error. The MAS consists of three agents and the output signal of each agent defines the percentage change of each gain. The increment or the reduction of each gain can be in the range of 0% to 100% of its initial value. The simulation results highlight the performance of the suggested hybrid control strategy through comparison with the conventional PID controller tuned by Z-N.

Sign in / Sign up

Export Citation Format

Share Document