Adaptive value function approximation for continuous-state stochastic dynamic programming

2013 ◽  
Vol 40 (4) ◽  
pp. 1076-1084 ◽  
Author(s):  
Huiyuan Fan ◽  
Prashant K. Tarun ◽  
Victoria C.P. Chen
Keyword(s):  
Dynamic Programming ◽  
Function Approximation ◽  
Stochastic Dynamic Programming ◽  
Value Function ◽  
Stochastic Dynamic ◽  
Value Function Approximation ◽  
Adaptive Value ◽  
Continuous State

Hierarchical fuzzy ART for Q-learning and its application in air combat simulation

2017 ◽  
Vol 08 (04) ◽  
pp. 1750052 ◽  
Author(s):  
Yanan Zhou ◽  
Yaofei Ma ◽  
Xiao Song ◽  
Guanghong Gong
Keyword(s):  
State Space ◽  
Function Approximation ◽  
Value Function ◽  
Early Stage ◽  
Network Parameter ◽  
Resonance Theory ◽  
Value Function Approximation ◽  
Fuzzy Art ◽  
Continuous State ◽  
Air Combat

Value function approximation plays an important role in reinforcement learning (RL) with continuous state space, which is widely used to build decision models in practice. Many traditional approaches require experienced designers to manually specify the formulization of the approximating function, leading to the rigid, non-adaptive representation of the value function. To address this problem, a novel Q-value function approximation method named ‘Hierarchical fuzzy Adaptive Resonance Theory’ (HiART) is proposed in this paper. HiART is based on the Fuzzy ART method and is an adaptive classification network that learns to segment the state space by classifying the training input automatically. HiART begins with a highly generalized structure where the number of the category nodes is limited, which is beneficial to speed up the learning process at the early stage. Then, the network is refined gradually by creating the attached sub-networks, and a layered network structure is formed during this process. Based on this adaptive structure, HiART alleviates the dependence on expert experience to design the network parameter. The effectiveness and adaptivity of HiART are demonstrated in the Mountain Car benchmark problem with both fast learning speed and low computation time. Finally, a simulation application example of the one versus one air combat decision problem illustrates the applicability of HiART.

Sign in / Sign up

Export Citation Format

Share Document