scholarly journals Adaptive Object Tracking via Multi-Angle Analysis Collaboration

Sensors ◽  
2018 ◽  
Vol 18 (11) ◽  
pp. 3606 ◽  
Author(s):  
Wanli Xue ◽  
Zhiyong Feng ◽  
Chao Xu ◽  
Zhaopeng Meng ◽  
Chengwei Zhang
Keyword(s):  
Object Tracking ◽  
Learning Algorithm ◽  
Action Space ◽  
Selection Strategy ◽  
Multiple Perspectives ◽  
Strategic Framework ◽  
Practical Applications ◽  
Q Learning ◽  
State Action ◽  
Speed And Accuracy

Although tracking research has achieved excellent performance in mathematical angles, it is still meaningful to analyze tracking problems from multiple perspectives. This motivation not only promotes the independence of tracking research but also increases the flexibility of practical applications. This paper presents a significant tracking framework based on the multi-dimensional state–action space reinforcement learning, termed as multi-angle analysis collaboration tracking (MACT). MACT is comprised of a basic tracking framework and a strategic framework which assists the former. Especially, the strategic framework is extensible and currently includes feature selection strategy (FSS) and movement trend strategy (MTS). These strategies are abstracted from the multi-angle analysis of tracking problems (observer’s attention and object’s motion). The content of the analysis corresponds to the specific actions in the multidimensional action space. Concretely, the tracker, regarded as an agent, is trained with Q-learning algorithm and ϵ -greedy exploration strategy, where we adopt a customized rewarding function to encourage robust object tracking. Numerous contrast experimental evaluations on the OTB50 benchmark demonstrate the effectiveness of the strategies and improvement in speed and accuracy of MACT tracker.

scholarly journals The New Geometric “State-Action” Space Representation for Q-Learning Algorithm for Protein Structure Folding Problem

2020 ◽  
pp. 59-73
Author(s):  
S. Chornozhuk
Keyword(s):  
Protein Structure ◽  
State Space ◽  
Learning Algorithm ◽  
Action Space ◽  
Space Representation ◽  
Q Learning ◽  
State Action ◽  
State Space Representation ◽  
Advantages And Disadvantages ◽  
Learning Techniques

Introduction. The spatial protein structure folding is an important and actual problem in computational biology. Considering the mathematical model of the task, it can be easily concluded that finding an optimal protein conformation in a three dimensional grid is a NP-hard problem. Therefore some reinforcement learning techniques such as Q-learning approach can be used to solve the problem. The article proposes a new geometric “state-action” space representation which significantly differs from all alternative representations used for this problem. The purpose of the article is to analyze existing approaches of different states and actions spaces representations for Q-learning algorithm for protein structure folding problem, reveal their advantages and disadvantages and propose the new geometric “state-space” representation. Afterwards the goal is to compare existing and the proposed approaches, make conclusions with also describing possible future steps of further research. Result. The work of the proposed algorithm is compared with others on the basis of 10 known chains with a length of 48 first proposed in [16]. For each of the chains the Q-learning algorithm with the proposed “state-space” representation outperformed the same Q-learning algorithm with alternative existing “state-space” representations both in terms of average and minimal energy values of resulted conformations. Moreover, a plenty of existing representations are used for a 2D protein structure predictions. However, during the experiments both existing and proposed representations were slightly changed or developed to solve the problem in 3D, which is more computationally demanding task. Conclusion. The quality of the Q-learning algorithm with the proposed geometric “state-action” space representation has been experimentally confirmed. Consequently, it’s proved that the further research is promising. Moreover, several steps of possible future research such as combining the proposed approach with deep learning techniques has been already suggested. Keywords: Spatial protein structure, combinatorial optimization, relative coding, machine learning, Q-learning, Bellman equation, state space, action space, basis in 3D space.

scholarly journals Implementation of modified Q learning technique in EMCAP control architecture

2017 ◽  
Vol 7 (1.5) ◽  
pp. 269
Author(s):  
D. Ganesha ◽  
Vijayakumar Maragal Venkatamuni
Keyword(s):  
Learning Algorithm ◽  
Control Strategies ◽  
Cognitive Architecture ◽  
Dynamic Environment ◽  
Learning System ◽  
Selection Strategy ◽  
Q Learning ◽  
Learning Techniques ◽  
Markov Decision ◽  
Environment Experiment

This research introduces a self learning modified (Q-Learning) techniques in a EMCAP (Enhanced Mind Cognitive Architecture of pupils). Q-learning is a modelless reinforcement learning (RL) methodology technique. In Specific, Q-learning can be applied to establish an optimal action-selection strategy for any respective Markov decision process. In this research introduces the modified Q-learning in a EMCAP (Enhanced Mind Cognitive Architecture of pupils). EMCAP architecture [1] enables and presents various agent control strategies for static and dynamic environment.  Experiment are conducted to evaluate the performace for each agent individually. For result comparison among different agent, the same statistics were collected. This work considered varied kind of agents in different level of architecture for experiment analysis. The Fungus world testbed has been considered for experiment which is has been implemented using SwI-Prolog 5.4.6. The fixed obstructs tend to be more versatile, to make a location that is specific to Fungus world testbed environment. The various parameters are introduced in an environment to test a agent’s performance.his modified q learning algorithm can be more suitable in EMCAP architecture.  The experiments are conducted the modified Q-Learning system gets more rewards compare to existing Q-learning.

A LEARNING ALGORITHM FOR DISCRETE-TIME STOCHASTIC CONTROL

2000 ◽  
Vol 14 (2) ◽  
pp. 243-258 ◽  
Author(s):  
V. S. Borkar
Keyword(s):  
Stochastic Control ◽  
Discrete Time ◽  
Learning Algorithm ◽  
Control Process ◽  
Almost Sure Convergence ◽  
Discrete State ◽  
Q Learning ◽  
State Action ◽  
Simulation Based ◽  
Action Spaces

A simulation-based algorithm for learning good policies for a discrete-time stochastic control process with unknown transition law is analyzed when the state and action spaces are compact subsets of Euclidean spaces. This extends the Q-learning scheme of discrete state/action problems along the lines of Baker [4]. Almost sure convergence is proved under suitable conditions.

scholarly journals Multi-agent cooperation Q-learning algorithm based on constrained Markov Game

2020 ◽  
Vol 17 (2) ◽  
pp. 647-664
Author(s):  
Yangyang Ge ◽  
Fei Zhu ◽  
Wei Huang ◽  
Peiyao Zhao ◽  
Quan Liu
Keyword(s):  
Reinforcement Learning ◽  
Learning Algorithm ◽  
Multi Agent System ◽  
Agent System ◽  
Action Function ◽  
Q Learning ◽  
State Action ◽  
Markov Game ◽  
Safety Constraints ◽  
Multi Agent

Multi-Agent system has broad application in real world, whose security performance, however, is barely considered. Reinforcement learning is one of the most important methods to resolve Multi-Agent problems. At present, certain progress has been made in applying Multi-Agent reinforcement learning to robot system, man-machine match, and automatic, etc. However, in the above area, an agent may fall into unsafe states where the agent may find it difficult to bypass obstacles, to receive information from other agents and so on. Ensuring the safety of Multi-Agent system is of great importance in the above areas where an agent may fall into dangerous states that are irreversible, causing great damage. To solve the safety problem, in this paper we introduce a Multi-Agent Cooperation Q-Learning Algorithm based on Constrained Markov Game. In this method, safety constraints are added to the set of actions, and each agent, when interacting with the environment to search for optimal values, should be restricted by the safety rules, so as to obtain an optimal policy that satisfies the security requirements. Since traditional Multi-Agent reinforcement learning algorithm is no more suitable for the proposed model in this paper, a new solution is introduced for calculating the global optimum state-action function that satisfies the safety constraints. We take advantage of the Lagrange multiplier method to determine the optimal action that can be performed in the current state based on the premise of linearizing constraint functions, under conditions that the state-action function and the constraint function are both differentiable, which not only improves the efficiency and accuracy of the algorithm, but also guarantees to obtain the global optimal solution. The experiments verify the effectiveness of the algorithm.

DIFFERENT LEARNING METHODOLOGIES FOR VISION-BASED NAVIGATION BEHAVIORS

2005 ◽  
Vol 19 (08) ◽  
pp. 949-975 ◽  
Author(s):  
G. CICIRELLI ◽  
T. D'ORAZIO ◽  
A. DISTANTE
Keyword(s):  
Visual Information ◽  
Learning Algorithm ◽  
Computational Cost ◽  
Fuzzy Model ◽  
System State ◽  
Continuous Action ◽  
Q Learning ◽  
State Action ◽  
Mobile Vehicle ◽  
Difficult Situations

In this work the complex behavior of localizing a mobile vehicle with respect to the door of the environment and then reaching the door has been developed. The robot uses visual information to detect and recognize the door and to determine its state with respect to it. This complex task has been divided into two separate behaviors: door-recognition and door-reaching. A supervised methodology based on learning by components has been applied for recognizing the door. Learning by components allows to recognize the door also in difficult situations such as partial occlusions and besides, it makes recognition independent of viewpoint variations and scale changes. An unsupervised methodology based on reinforcement learning has been used for the door-reaching behavior, instead. The image of the door gives information about the relative position of the vehicle with respect to the door. Then the Q-learning algorithm is used to generate the optimal state-action associations. The problem of defining the state and the action sets has been addressed with the aim of producing smooth paths, of reducing the effects of visual errors during real navigation, and of keeping low the computational cost during the learning phase. A novel way to obtain a continuous action set has been introduced: it uses a fuzzy model to evaluate the system state. Experimental results in real environment show both the robustness of the door-recognition behavior and the generality of the door-reaching behavior.

Ethology-Based Approximate Adaptive Learning

2012 ◽  
pp. 1856-1878
Author(s):  
James F. Peters ◽  
Shabnam Shahfar
Keyword(s):  
Machine Learning ◽  
Learning Theory ◽  
Adaptive Learning ◽  
Learning Algorithm ◽  
Near Sets ◽  
Practical Applications ◽  
Q Learning ◽  
Changing Environments ◽  
Complete Framework ◽  
Observed Behavior

The problem considered in this chapter is how to use the observed behavior of organisms as a basis for machine learning. The proposed approach for machine learning combines near sets and ethology. It leads to novel forms of Q-learning algorithm that have practical applications in the controlling the behavior of machines, which learn to adapt to changing environments. Both traditional and new forms of adaptive learning theory and applications are considered in this chapter. A complete framework for an ethology-based approximate adaptive learning is established by using near sets.

Ethology-Based Approximate Adaptive Learning

2011 ◽  
pp. 469-494
Author(s):  
James F. Peters ◽  
Shabnam Shahfar
Keyword(s):  
Machine Learning ◽  
Learning Theory ◽  
Adaptive Learning ◽  
Learning Algorithm ◽  
Near Sets ◽  
Practical Applications ◽  
Q Learning ◽  
Changing Environments ◽  
Complete Framework ◽  
Observed Behavior

The problem considered in this chapter is how to use the observed behavior of organisms as a basis for machine learning. The proposed approach for machine learning combines near sets and ethology. It leads to novel forms of Q-learning algorithm that have practical applications in the controlling the behavior of machines, which learn to adapt to changing environments. Both traditional and new forms of adaptive learning theory and applications are considered in this chapter. A complete framework for an ethology-based approximate adaptive learning is established by using near sets.

Neural Q-Learning Based Mobile Robot Navigation

2012 ◽  
Vol 433-440 ◽  
pp. 721-726
Author(s):  
Soh Chin Yun ◽  
S. Parasuraman ◽  
Velappa Ganapathy ◽  
Halim Kusuma Joe
Keyword(s):  
Neural Network ◽  
Learning Algorithm ◽  
Mobile Robot Navigation ◽  
Matlab Simulation ◽  
Training Process ◽  
Q Learning ◽  
State Action ◽  
Training Samples ◽  
Artificial Neural Network Ann ◽  
Trained Neural Network

This research is focused on the integration of multi-layer Artificial Neural Network (ANN) and Q-Learning to perform online learning control. In the first learning phase, the agent explores the unknown surroundings and gathers state-action information through the unsupervised Q-Learning algorithm. Second training process involves ANN which utilizes the state-action information gathered in the earlier phase of training samples. During final application of the controller, Q-Learning would be used as primary navigating tool whereas the trained Neural Network will be employed when approximation is needed. MATLAB simulation was developed to verify and the algorithm was validated in real-time using Team AmigoBotTM robot. The results obtained from both simulation and real world experiments are discussed.

scholarly journals Improved Q -Learning Method for Multirobot Formation and Path Planning with Concave Obstacles

2021 ◽  
Vol 2021 ◽  
pp. 1-14
Author(s):  
Zhilin Fan ◽  
Fei Liu ◽  
Xinshun Ning ◽  
Yilin Han ◽  
Jian Wang ◽  
...  
Keyword(s):  
Path Planning ◽  
Learning Algorithm ◽  
Convergence Time ◽  
Multirobot Systems ◽  
Selection Strategy ◽  
Q Learning ◽  
Unknown Environment ◽  
Traditional Algorithm ◽  
Leader Following ◽  
Tracking Strategy

Aiming at the formation and path planning of multirobot systems in an unknown environment, a path planning method for multirobot formation based on improved Q -learning is proposed. Based on the leader-following approach, the leader robot uses an improved Q -learning algorithm to plan the path and the follower robot achieves a tracking strategy of gravitational potential field (GPF) by designing a cost function to select actions. Specifically, to improve the Q-learning, Q -value is initialized by environmental guidance of the target’s GPF. Then, the virtual obstacle-filling avoidance strategy is presented to fill non-obstacles which is judged to tend to concave obstacles with virtual obstacles. Besides, the simulated annealing (SA) algorithm whose controlling temperature is adjusted in real time according to the learning situation of the Q -learning is applied to improve the action selection strategy. The experimental results show that the improved Q -learning algorithm reduces the convergence time by 89.9% and the number of convergence rounds by 63.4% compared with the traditional algorithm. With the help of the method, multiple robots have a clear division of labor and quickly plan a globally optimized formation path in a completely unknown environment.

Sign in / Sign up

Export Citation Format

Share Document