Multistability of recurrent neural networks with time-varying delays and nonincreasing activation function

2016 ◽  
Vol 216 ◽  
pp. 135-142 ◽  
Author(s):  
Fanghai Zhang ◽  
Zhigang Zeng
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Activation Function ◽  
Time Varying

scholarly journals Improved Stability Criteria of Static Recurrent Neural Networks with a Time-Varying Delay

2014 ◽  
Vol 2014 ◽  
pp. 1-7 ◽  
Author(s):  
Lei Ding ◽  
Hong-Bing Zeng ◽  
Wei Wang ◽  
Fei Yu
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Linear Matrix ◽  
Time Varying ◽  
Time Varying Delay ◽  
Improved Stability ◽  
The Stability ◽  
Delay Dependent ◽  
Delay Dependent Stability ◽  
Varying Delay

This paper investigates the stability of static recurrent neural networks (SRNNs) with a time-varying delay. Based on the complete delay-decomposing approach and quadratic separation framework, a novel Lyapunov-Krasovskii functional is constructed. By employing a reciprocally convex technique to consider the relationship between the time-varying delay and its varying interval, some improved delay-dependent stability conditions are presented in terms of linear matrix inequalities (LMIs). Finally, a numerical example is provided to show the merits and the effectiveness of the proposed methods.

Recurrent Neural Networks with Small Weights Implement Definite Memory Machines

2003 ◽  
Vol 15 (8) ◽  
pp. 1897-1929 ◽  
Author(s):  
Barbara Hammer ◽  
Peter Tiňo
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Experimental Studies ◽  
Activation Function ◽  
Transition Function ◽  
Recurrent Network ◽  
Point Of View ◽  
Recurrent Networks ◽  
Contraction Parameter ◽  
Arbitrary Precision

Recent experimental studies indicate that recurrent neural networks initialized with “small” weights are inherently biased toward definite memory machines (Tiňno, Čerňanský, & Beňušková, 2002a, 2002b). This article establishes a theoretical counterpart: transition function of recurrent network with small weights and squashing activation function is a contraction. We prove that recurrent networks with contractive transition function can be approximated arbitrarily well on input sequences of unbounded length by a definite memory machine. Conversely, every definite memory machine can be simulated by a recurrent network with contractive transition function. Hence, initialization with small weights induces an architectural bias into learning with recurrent neural networks. This bias might have benefits from the point of view of statistical learning theory: it emphasizes one possible region of the weight space where generalization ability can be formally proved. It is well known that standard recurrent neural networks are not distribution independent learnable in the probably approximately correct (PAC) sense if arbitrary precision and inputs are considered. We prove that recurrent networks with contractive transition function with a fixed contraction parameter fulfill the so-called distribution independent uniform convergence of empirical distances property and hence, unlike general recurrent networks, are distribution independent PAC learnable.

Sign in / Sign up

Export Citation Format

Share Document