Improvement of pattern learning and recognition capability in ratio-memory cellular neural networks with non-discrete-type Hebbian learning algorithm

2003 ◽  
Author(s):  
Chung-Yu Wu ◽  
Jui-Lin Lai
Keyword(s):  
Neural Networks ◽  
Hebbian Learning ◽  
Learning Algorithm ◽  
Cellular Neural Networks ◽  
Pattern Learning ◽  
Recognition Capability ◽  
Discrete Type

scholarly journals GENERALIZED CELLULAR NEURAL NETWORKS (GCNNs) CONSTRUCTED USING PARTICLE SWARM OPTIMIZATION FOR SPATIO-TEMPORAL EVOLUTIONARY PATTERN IDENTIFICATION

2008 ◽  
Vol 18 (12) ◽  
pp. 3611-3624 ◽  
Author(s):  
H. L. WEI ◽  
S. A. BILLINGS
Keyword(s):  
Neural Networks ◽  
Particle Swarm Optimization ◽  
Orthogonal Projection ◽  
Learning Algorithm ◽  
Projection Pursuit ◽  
Particle Swarm ◽  
Cellular Neural Networks ◽  
Modeling Framework ◽  
Swarm Optimization ◽  
Spatio Temporal

Particle swarm optimization (PSO) is introduced to implement a new constructive learning algorithm for training generalized cellular neural networks (GCNNs) for the identification of spatio-temporal evolutionary (STE) systems. The basic idea of the new PSO-based learning algorithm is to successively approximate the desired signal by progressively pursuing relevant orthogonal projections. This new algorithm will thus be referred to as the orthogonal projection pursuit (OPP) algorithm, which is in mechanism similar to the conventional projection pursuit approach. A novel two-stage hybrid training scheme is proposed for constructing a parsimonious GCNN model. In the first stage, the orthogonal projection pursuit algorithm is applied to adaptively and successively augment the network, where adjustable parameters of the associated units are optimized using a particle swarm optimizer. The resultant network model produced at the first stage may be redundant. In the second stage, a forward orthogonal regression (FOR) algorithm, aided by mutual information estimation, is applied to refine and improve the initially trained network. The effectiveness and performance of the proposed method is validated by applying the new modeling framework to a spatio-temporal evolutionary system identification problem.

How Inhibitory Oscillations Can Train Neural Networks and Punish Competitors

2006 ◽  
Vol 18 (7) ◽  
pp. 1577-1610 ◽  
Author(s):  
Kenneth A. Norman ◽  
Ehren Newman ◽  
Greg Detre ◽  
Sean Polyn
Keyword(s):  
Neural Networks ◽  
Weight Change ◽  
Hebbian Learning ◽  
Learning Algorithm ◽  
Large Numbers ◽  
Neural Inhibition ◽  
New Learning

We present a new learning algorithm that leverages oscillations in the strength of neural inhibition to train neural networks. Raising inhibition can be used to identify weak parts of target memories, which are then strengthened. Conversely, lowering inhibition can be used to identify competitors, which are then weakened. To update weights, we apply the Contrastive Hebbian Learning equation to successive time steps of the network. The sign of the weight change equation varies as a function of the phase of the inhibitory oscillation. We show that the learning algorithm can memorize large numbers of correlated input patterns without collapsing and that it shows good generalization to test patterns that do not exactly match studied patterns.

Sign in / Sign up

Export Citation Format

Share Document