Adaptive function approximation in reinforcement learning with an interpolating growing neural gas1

2013 ◽  
Vol 11 (1) ◽  
pp. 55-69
Author(s):  
Michael Baumann ◽  
Hans Kleine Büning
Keyword(s):  
Reinforcement Learning ◽  
Function Approximation ◽  
Adaptive Function

Position and Attitude Control of Underactuated Drones Using the Adaptive Function Approximation Technique

2020 ◽  
Author(s):  
Azin Shamshirgaran ◽  
Donald Ebeigbe ◽  
Dan Simon
Keyword(s):  
Disturbance Rejection ◽  
Function Approximation ◽  
Control Method ◽  
System Stability ◽  
Linear Feedback ◽  
Approximation Technique ◽  
Adaptive Function ◽  
Edge Tracking ◽  
Highly Nonlinear ◽  
Function Approximation Technique

Abstract Despite the popularity of drones and their relatively simple operation, the underlying control algorithms can be difficult to design due to the drones’ underactuation and highly nonlinear properties. This paper focuses on position and orientation control of drones to address challenges such as path and edge tracking, and disturbance rejection. The adaptive function approximation technique control method is used to control an underactuated and nonlinear drone. The controller utilizes reference attitude signals, that are derived from a proportional derivative (PD) linear feedback control methodology. To avoid analytic expressions for the reference attitude velocities, we employ a continuous-time Kalman filter based on a model of the measurement signal — which is derived by passing the reference attitude position through a low-pass signal differentiator — as a second-order Newtonian system. Stability of the closed loop system is proven using a Lyapunov function. Our design methodology simplifies the control process by requiring only a few tuning variables, while being robust to time-varying and time-invariant uncertainties with unknown variation bounds, and avoids the requirement for the knowledge of the dynamic equation that governs the attitude of the drone. Three different scenarios are simulated and our control method shows better accuracy than the proportional-derivative controller in terms of edge tracking and disturbance rejection.

FUZZY STATE AGGREGATION AND POLICY HILL CLIMBING FOR STOCHASTIC ENVIRONMENTS

2006 ◽  
Vol 06 (03) ◽  
pp. 413-428 ◽  
Author(s):  
DEAN C. WARDELL ◽  
GILBERT L. PETERSON
Keyword(s):  
Reinforcement Learning ◽  
Function Approximation ◽  
Hill Climbing ◽  
Individual Agent ◽  
Q Learning ◽  
Stochastic Environments ◽  
State Aggregation ◽  
Multi Agent ◽  
Fuzzy State ◽  
Better Than

Reinforcement learning is one of the more attractive machine learning technologies, due to its unsupervised learning structure and ability to continually learn even as the operating environment changes. Additionally, by applying reinforcement learning to multiple cooperative software agents (a multi-agent system) not only allows each individual agent to learn from its own experience, but also opens up the opportunity for the individual agents to learn from the other agents in the system, thus accelerating the rate of learning. This research presents the novel use of fuzzy state aggregation, as the means of function approximation, combined with the fastest policy hill climbing methods of Win or Lose Fast (WoLF) and policy-dynamics based WoLF (PD-WoLF). The combination of fast policy hill climbing and fuzzy state aggregation function approximation is tested in two stochastic environments: Tileworld and the simulated robot soccer domain, RoboCup. The Tileworld results demonstrate that a single agent using the combination of FSA and PHC learns quicker and performs better than combined fuzzy state aggregation and Q-learning reinforcement learning alone. Results from the multi-agent RoboCup domain again illustrate that the policy hill climbing algorithms perform better than Q-learning alone in a multi-agent environment. The learning is further enhanced by allowing the agents to share their experience through a weighted strategy sharing.

Sign in / Sign up

Export Citation Format

Share Document