Dead-zone Kalman filter algorithm for recurrent neural networks

2006 ◽  
Author(s):  
J. de Jesus Rubio ◽  
Wen Yu
Keyword(s):  
Neural Networks ◽  
Kalman Filter ◽  
Recurrent Neural Networks ◽  
Dead Zone ◽  
Kalman Filter Algorithm

An Augmented Extended Kalman Filter Algorithm for Complex-Valued Recurrent Neural Networks

2007 ◽  
Vol 19 (4) ◽  
pp. 1039-1055 ◽  
Author(s):  
Su Lee Goh ◽  
Danilo P. Mandic
Keyword(s):  
Neural Networks ◽  
Kalman Filter ◽  
Extended Kalman Filter ◽  
Recurrent Neural Networks ◽  
Adaptive Filters ◽  
General Complex ◽  
Recent Developments ◽  
Kalman Filter Algorithm ◽  
Fully Connected ◽  
Complex Valued

An augmented complex-valued extended Kalman filter (ACEKF) algorithm for the class of nonlinear adaptive filters realized as fully connected recurrent neural networks is introduced. This is achieved based on some recent developments in the so-called augmented complex statistics and the use of general fully complex nonlinear activation functions within the neurons. This makes the ACEKF suitable for processing general complex-valued nonlinear and nonstationary signals and also bivariate signals with strong component correlations. Simulations on benchmark and real-world complex-valued signals support the approach.

Extended Kalman Filter–Based Pruning Method for Recurrent Neural Networks

1998 ◽  
Vol 10 (6) ◽  
pp. 1481-1505 ◽  
Author(s):  
John Sum ◽  
Lai-wan Chan ◽  
Chi-sing Leung ◽  
Gilbert H. Young
Keyword(s):  
Neural Networks ◽  
Time Series ◽  
Kalman Filter ◽  
Extended Kalman Filter ◽  
Recurrent Neural Networks ◽  
Posterior Probability ◽  
Linear Time ◽  
Learning Methods ◽  
Pruning Method ◽  
Gradient Based

Pruning is one of the effective techniques for improving the generalization error of neural networks. Existing pruning techniques are derived mainly from the viewpoint of energy minimization, which is commonly used in gradient-based learning methods. In recurrent networks, extended Kalman filter (EKF)–based training has been shown to be superior to gradient-based learning methods in terms of speed. This article explains a pruning procedure for recurrent neural networks using EKF training. The sensitivity of a posterior probability is used as a measure of the importance of a weight instead of error sensitivity since posterior probability density is readily obtained from this training method. The pruning procedure is tested using three problems: (1) the prediction of a simple linear time series, (2) the identification of a nonlinear system, and (3) the prediction of an exchange-rate time series. Simulation results demonstrate that the proposed pruning method is able to reduce the number of parameters and improve the generalization ability of a recurrent network.

Sign in / Sign up

Export Citation Format

Share Document