scholarly journals Local online learning in recurrent networks with random feedback

2018 ◽  
Author(s):  
James M. Murray
Keyword(s):  
Online Learning ◽  
Computational Neuroscience ◽  
Learning Rule ◽  
Recurrent Networks ◽  
Biological Features ◽  
Mathematical Arguments ◽  
Learning Rules ◽  
New Learning ◽  
Gradient Based ◽  
The Brain

AbstractA longstanding challenge for computational neuroscience has been the development of biologically plausible learning rules for recurrent neural networks (RNNs) enabling the production and processing of time-dependent signals such as those that might drive movement or facilitate working memory. Classic gradient-based algorithms for training RNNs have been available for decades, but they are inconsistent with known biological features of the brain, such as causality and locality. In this work we derive an approximation to gradient-based learning that comports with these biologically motivated constraints. Specifically, the online learning rule for the synaptic weights involves only local information about the pre- and postsynaptic activities, in addition to a random feedback projection of the RNN output error. In addition to providing mathematical arguments for the effectiveness of the new learning rule, we show through simulations that it can be used to train an RNN to successfully perform a variety of tasks. Finally, to overcome the difficulty of training an RNN over a very large number of timesteps, we propose an augmented circuit architecture that allows the RNN to concatenate short-duration patterns into sequences of longer duration.

scholarly journals Local online learning in recurrent networks with random feedback

eLife ◽  
2019 ◽  
Vol 8 ◽  
Author(s):  
James M Murray
Keyword(s):  
Synaptic Weight ◽  
Learning Rule ◽  
Local Information ◽  
Recurrent Networks ◽  
Biological Features ◽  
Mathematical Arguments ◽  
Large Numbers ◽  
New Learning ◽  
Gradient Based ◽  
The Brain

Recurrent neural networks (RNNs) enable the production and processing of time-dependent signals such as those involved in movement or working memory. Classic gradient-based algorithms for training RNNs have been available for decades, but are inconsistent with biological features of the brain, such as causality and locality. We derive an approximation to gradient-based learning that comports with these constraints by requiring synaptic weight updates to depend only on local information about pre- and postsynaptic activities, in addition to a random feedback projection of the RNN output error. In addition to providing mathematical arguments for the effectiveness of the new learning rule, we show through simulations that it can be used to train an RNN to perform a variety of tasks. Finally, to overcome the difficulty of training over very large numbers of timesteps, we propose an augmented circuit architecture that allows the RNN to concatenate short-duration patterns into longer sequences.

scholarly journals A solution to the learning dilemma for recurrent networks of spiking neurons

2019 ◽  
Author(s):  
Guillaume Bellec ◽  
Franz Scherr ◽  
Anand Subramoney ◽  
Elias Hajek ◽  
Darjan Salaj ◽  
...  
Keyword(s):  
Recurrent Neural Networks ◽  
Energy Efficient ◽  
Gradient Descent ◽  
Spiking Neurons ◽  
Recurrent Networks ◽  
Network Learning ◽  
Backpropagation Through Time ◽  
New Learning ◽  
On Chip ◽  
The Brain

AbstractRecurrently connected networks of spiking neurons underlie the astounding information processing capabilities of the brain. But in spite of extensive research, it has remained open how they can learn through synaptic plasticity to carry out complex network computations. We argue that two pieces of this puzzle were provided by experimental data from neuroscience. A new mathematical insight tells us how these pieces need to be combined to enable biologically plausible online network learning through gradient descent, in particular deep reinforcement learning. This new learning method – called e-prop – approaches the performance of BPTT (backpropagation through time), the best known method for training recurrent neural networks in machine learning. In addition, it suggests a method for powerful on-chip learning in novel energy-efficient spike-based hardware for AI.

A Learning Rule for the Emergence of Stable Dynamics and Timing in Recurrent Networks

2005 ◽  
Vol 94 (4) ◽  
pp. 2275-2283 ◽  
Author(s):  
Dean V. Buonomano
Keyword(s):  
Time Window ◽  
Learning Rule ◽  
Neural Dynamics ◽  
Neural Processing ◽  
Recurrent Networks ◽  
Spike Time ◽  
Cortical Networks ◽  
Direct Function ◽  
Learning Rules ◽  
Timing Mechanisms

Neural dynamics within recurrent cortical networks is an important component of neural processing. However, the learning rules that allow networks composed of hundreds or thousands of recurrently connected neurons to develop stable dynamical states are poorly understood. Here I use a neural network model to examine the emergence of stable dynamical states within recurrent networks. I describe a learning rule that can account both for the development of stable dynamics and guide networks to states that have been observed experimentally, specifically, states that instantiate a sparse code for time. Across trials, each neuron fires during a specific time window; by connecting the neurons to a hypothetical set of output units, it is possible to generate arbitrary spatial-temporal output patterns. Intertrial jitter of the spike time of a given neuron increases as a direct function of the delay at which it fires. These results establish a learning rule by which cortical networks can potentially process temporal information in a self-organizing manner, in the absence of specialized timing mechanisms.

scholarly journals Unpredictability vs. volatility and the control of learning

2020 ◽  
Author(s):  
Payam Piray ◽  
Nathaniel D Daw
Keyword(s):  
Anxiety Disorders ◽  
Computational Neuroscience ◽  
Rate Of Change ◽  
Learning Rate ◽  
Compensatory Behaviors ◽  
Learning Rates ◽  
New Learning ◽  
Two Factors ◽  
Show Evidence ◽  
The Brain

Influential research in computational neuroscience has stressed the importance of uncertainty for controlling the speed of learning, and of volatility, i.e. the inferred rate of change, in this process. Here, we investigate a neglected feature of these models: learning rates are jointly determined by the comparison between volatility and a second factor, unpredictability, which reflects moment-to-moment stochasticity. Like volatility, unpredictability can vary and must be estimated by the learner, but much previous research has focused on estimation of volatility while unpredictability is assumed fixed and known. We introduce a new learning model, in which both factors are learned from experience. We show evidence from behavioral neuroscience that the brain distinguishes these two factors and adjusts the learning rate accordingly. The model highlights the interdependency in inferences about volatility and unpredictability, which leads it to paradoxical compensatory behaviors if inference about either factor is damaged. This provides a novel mechanism for understanding pathological learning in amygdala damage and anxiety disorders.

scholarly journals Weight perturbation learning outperforms node perturbation on broad classes of temporally extended tasks

2021 ◽  
Author(s):  
Paul Manfred Züge ◽  
Christian Klos ◽  
Raoul-Martin Memmesheimer
Keyword(s):  
Biologically Relevant ◽  
Network Learning ◽  
Learning Tasks ◽  
Learning Rules ◽  
New Learning ◽  
Error Dynamics ◽  
Low Dimensional ◽  
Task Types ◽  
The Brain ◽  
Better Than

Biological constraints often impose restrictions for plausible plasticity rules such as locality and reward-based rather than supervised learning. Two learning rules that comply with these restrictions are weight (WP) and node (NP) perturbation. NP is often used in learning studies, in particular as a benchmark; it is considered to be superior to WP and more likely neurobiologically realized, as the number of weights and therefore their perturbation dimension typically massively exceed the number of nodes. Here we show that this conclusion no longer holds when we take two biologically relevant properties into account: First, tasks extend in time. This increases the perturbation dimension of NP but not WP. Second, tasks are low dimensional, with many weight configurations providing solutions. We analytically delineate regimes where these properties let WP perform as well as or better than NP. Further we find qualitative features of the weight and error dynamics that allow to distinguish which of the rules underlie a learning process: in WP, but not NP, weights mediating zero input diffuse and gathering batches of subtasks in a trial decreases the number of trials required. The insights suggest new learning rules, which combine for specific task types the advantages of WP and NP. Using numerical simulations, we generalize the results to networks with various architectures solving biologically relevant and standard network learning tasks. Our findings suggest WP and NP as similarly plausible candidates for learning in the brain and as similarly important benchmarks.

scholarly journals Contrastive Learning and Neural Oscillations

1991 ◽  
Vol 3 (4) ◽  
pp. 526-545 ◽  
Author(s):  
Pierre Baldi ◽  
Fernando Pineda
Keyword(s):  
Gradient Descent ◽  
Learning Algorithms ◽  
Learning Rule ◽  
Unified Framework ◽  
Contrast Function ◽  
New Learning ◽  
Different Types ◽  
Free Network ◽  
Two Phases ◽  
The Brain

The concept of Contrastive Learning (CL) is developed as a family of possible learning algorithms for neural networks. CL is an extension of Deterministic Boltzmann Machines to more general dynamical systems. During learning, the network oscillates between two phases. One phase has a teacher signal and one phase has no teacher signal. The weights are updated using a learning rule that corresponds to gradient descent on a contrast function that measures the discrepancy between the free network and the network with a teacher signal. The CL approach provides a general unified framework for developing new learning algorithms. It also shows that many different types of clamping and teacher signals are possible. Several examples are given and an analysis of the landscape of the contrast function is proposed with some relevant predictions for the CL curves. An approach that may be suitable for collective analog implementations is described. Simulation results and possible extensions are briefly discussed together with a new conjecture regarding the function of certain oscillations in the brain. In the appendix, we also examine two extensions of contrastive learning to time-dependent trajectories.

Perception

2018 ◽  
Author(s):  
Joel Z. Leibo ◽  
Tomaso Poggio
Keyword(s):  
Computational Neuroscience ◽  
Feature Detection ◽  
Pedestrian Detection ◽  
Detection Systems ◽  
Catching Up ◽  
Feature Detectors ◽  
Recognition Systems ◽  
Engineered Systems ◽  
Perceptual Systems ◽  
The Brain

This chapter provides an overview of biological perceptual systems and their underlying computational principles focusing on the sensory sheets of the retina and cochlea and exploring how complex feature detection emerges by combining simple feature detectors in a hierarchical fashion. We also explore how the microcircuits of the neocortex implement such schemes pointing out similarities to progress in the field of machine vision driven deep learning algorithms. We see signs that engineered systems are catching up with the brain. For example, vision-based pedestrian detection systems are now accurate enough to be installed as safety devices in (for now) human-driven vehicles and the speech recognition systems embedded in smartphones have become increasingly impressive. While not being entirely biologically based, we note that computational neuroscience, as described in this chapter, makes up a considerable portion of such systems’ intellectual pedigree.

ENERGY RELAXATION FOR HOPFIELD NETWORK WITH THE NEW LEARNING RULE

2009 ◽  
Author(s):  
Saratha Sathasivam ◽  
Abdul Halim Hakim ◽  
Pandian Vasant ◽  
Nader Barsoum
Keyword(s):  
Learning Rule ◽  
Energy Relaxation ◽  
Hopfield Network ◽  
New Learning
Sign in / Sign up

Export Citation Format

Share Document