scholarly journals Artificial Dendritic Neurons Enable Self-Supervised Temporal Feature Extraction

2019 ◽  
Author(s):  
Toshitake Asabuki ◽  
Tomoki Fukai
Keyword(s):  
Hebbian Learning ◽  
Learning Rule ◽  
Conditional Entropy ◽  
Fundamental Principle ◽  
Neuron Models ◽  
Learning Tasks ◽  
Conventional Machine ◽  
Characteristic Features ◽  
Temporal Events ◽  
Information Streams

AbstractThe brain identifies potentially salient features within continuous information streams to appropriately process external and internal temporal events. This requires the compression or abstraction of information streams, for which no effective information principles are known. Here, we propose conditional entropy minimization learning as the fundamental principle of such temporal processing. We show that this learning rule resembles Hebbian learning with backpropagating action potentials in dendritic neuron models. Moreover, networks of the dendritic neurons can perform a surprisingly wide variety of complex unsupervised learning tasks. Our model not only accounts for the mechanisms of chunking of temporal inputs in the human brain but also accomplishes blind source separation of correlated mixed signals, which cannot be solved by conventional machine learning methods, such as independent-component analysis.One Sentence SummaryNeurons use soma-dendrite interactions to self-supervise the learning of characteristic features of various temporal inputs.

scholarly journals Sparse Coding with a Somato-Dendritic Rule

2018 ◽  
Author(s):  
Damien Drix ◽  
Verena V. Hafner ◽  
Michael Schmuker
Keyword(s):  
Sparse Coding ◽  
Cortical Neurons ◽  
Hebbian Learning ◽  
Learning Rule ◽  
Homeostatic Plasticity ◽  
Hybrid Network ◽  
Neuron Models ◽  
Functional Interpretation ◽  
Inhibitory Plasticity ◽  
Somatic Inhibition

AbstractCortical neurons are silent most of the time. This sparse activity is energy efficient, and the resulting neural code has favourable properties for associative learning. Most neural models of sparse coding use some form of homeostasis to ensure that each neuron fires infrequently. But homeostatic plasticity acting on a fast timescale may not be biologically plausible, and could lead to catastrophic forgetting in embodied agents that learn continuously. We set out to explore whether inhibitory plasticity could play that role instead, regulating both the population sparseness and the average firing rates. We put the idea to the test in a hybrid network where rate-based dendritic compartments integrate the feedforward input, while spiking somas compete through recurrent inhibition. A somato-dendritic learning rule allows somatic inhibition to modulate nonlinear Hebbian learning in the dendrites. Trained on MNIST digits and natural images, the network discovers independent components that form a sparse encoding of the input and support linear decoding. These findings con-firm that intrinsic plasticity is not strictly required for regulating sparseness: inhibitory plasticity can have the same effect, although that mechanism comes with its own stability-plasticity dilemma. Going beyond point neuron models, the network illustrates how a learning rule can make use of dendrites and compartmentalised inputs; it also suggests a functional interpretation for clustered somatic inhibition in cortical neurons.

scholarly journals Characteristics of sequential activity in networks with temporally asymmetric Hebbian learning

2020 ◽  
Vol 117 (47) ◽  
pp. 29948-29958
Author(s):  
Maxwell Gillett ◽  
Ulises Pereira ◽  
Nicolas Brunel
Keyword(s):  
Storage Capacity ◽  
Synaptic Weight ◽  
Hebbian Learning ◽  
Mean Field Theory ◽  
Mean Field ◽  
Learning Rule ◽  
Network Activity ◽  
Synaptic Connectivity ◽  
Random Input ◽  
Low Dimensional

Sequential activity has been observed in multiple neuronal circuits across species, neural structures, and behaviors. It has been hypothesized that sequences could arise from learning processes. However, it is still unclear whether biologically plausible synaptic plasticity rules can organize neuronal activity to form sequences whose statistics match experimental observations. Here, we investigate temporally asymmetric Hebbian rules in sparsely connected recurrent rate networks and develop a theory of the transient sequential activity observed after learning. These rules transform a sequence of random input patterns into synaptic weight updates. After learning, recalled sequential activity is reflected in the transient correlation of network activity with each of the stored input patterns. Using mean-field theory, we derive a low-dimensional description of the network dynamics and compute the storage capacity of these networks. Multiple temporal characteristics of the recalled sequential activity are consistent with experimental observations. We find that the degree of sparseness of the recalled sequences can be controlled by nonlinearities in the learning rule. Furthermore, sequences maintain robust decoding, but display highly labile dynamics, when synaptic connectivity is continuously modified due to noise or storage of other patterns, similar to recent observations in hippocampus and parietal cortex. Finally, we demonstrate that our results also hold in recurrent networks of spiking neurons with separate excitatory and inhibitory populations.

scholarly journals Synaptic memory devices from CoO/Nb:SrTiO 3 junction

2019 ◽  
Vol 6 (4) ◽  
pp. 181098 ◽  
Author(s):  
Le Zhao ◽  
Jie Xu ◽  
Xiantao Shang ◽  
Xue Li ◽  
Qiang Li ◽  
...  
Keyword(s):  
Large Scale ◽  
Hebbian Learning ◽  
Learning Rule ◽  
Charge Trapping ◽  
Spike Timing ◽  
Hebbian Learning Rule ◽  
Synaptic Functions ◽  
Neuromorphic System ◽  
Reduce Energy Consumption ◽  
Novel Design

Non-volatile memristors are promising for future hardware-based neurocomputation application because they are capable of emulating biological synaptic functions. Various material strategies have been studied to pursue better device performance, such as lower energy cost, better biological plausibility, etc. In this work, we show a novel design for non-volatile memristor based on CoO/Nb:SrTiO 3 heterojunction. We found the memristor intrinsically exhibited resistivity switching behaviours, which can be ascribed to the migration of oxygen vacancies and charge trapping and detrapping at the heterojunction interface. The carrier trapping/detrapping level can be finely adjusted by regulating voltage amplitudes. Gradual conductance modulation can therefore be realized by using proper voltage pulse stimulations. And the spike-timing-dependent plasticity, an important Hebbian learning rule, has been implemented in the device. Our results indicate the possibility of achieving artificial synapses with CoO/Nb:SrTiO 3 heterojunction. Compared with filamentary type of the synaptic device, our device has the potential to reduce energy consumption, realize large-scale neuromorphic system and work more reliably, since no structural distortion occurs.

ASSOCIATIVE MEMORY IN NEURAL NETWORKS WITH THE HEBBIAN LEARNING RULE

1989 ◽  
Vol 03 (07) ◽  
pp. 555-560 ◽  
Author(s):  
M.V. TSODYKS
Keyword(s):  
Neural Networks ◽  
Associative Memory ◽  
Hebbian Learning ◽  
Learning Rule ◽  
Network Activity ◽  
Hebbian Learning Rule ◽  
Level Of Activity ◽  
The Mean ◽  
Full Network ◽  
Simultaneous Activity

We consider the Hopfield model with the most simple form of the Hebbian learning rule, when only simultaneous activity of pre- and post-synaptic neurons leads to modification of synapse. An extra inhibition proportional to full network activity is needed. Both symmetric nondiluted and asymmetric diluted networks are considered. The model performs well at extremely low level of activity p<K−1/2, where K is the mean number of synapses per neuron.

Growing Self-Organizing Surface Map: Learning a Surface Topology from a Point Cloud

2010 ◽  
Vol 22 (3) ◽  
pp. 689-729 ◽  
Author(s):  
Vilson Luiz Dalle Mole ◽  
Aluizio Fausto Ribeiro Araújo
Keyword(s):  
Point Cloud ◽  
Hebbian Learning ◽  
Learning Rule ◽  
Surface Topology ◽  
Map Learning ◽  
3D Space ◽  
Dense Point ◽  
Equilateral Triangles ◽  
Sample Presentation ◽  
Self Organizing

The growing self-organizing surface map (GSOSM) is a novel map model that learns a folded surface immersed in a 3D space. Starting from a dense point cloud, the surface is reconstructed through an incremental mesh composed of approximately equilateral triangles. Unlike other models such as neural meshes (NM), the GSOSM builds a surface topology while accepting any sequence of sample presentation. The GSOSM model introduces a novel connection learning rule called competitive connection Hebbian learning (CCHL), which produces a complete triangulation. GSOSM reconstructions are accurate and often free of false or overlapping faces. This letter presents and discusses the GSOSM model. It also presents and analyzes a set of results and compares GSOSM with some other models.

How the Shape of Pre- and Postsynaptic Signals Can Influence STDP: A Biophysical Model

2004 ◽  
Vol 16 (3) ◽  
pp. 595-625 ◽  
Author(s):  
Ausra Saudargiene ◽  
Bernd Porr ◽  
Florentin Wörgötter
Keyword(s):  
Membrane Potential ◽  
Weight Change ◽  
Hebbian Learning ◽  
Dendritic Tree ◽  
Learning Rule ◽  
Long Term Potentiation ◽  
Spike Timing ◽  
Biophysical Model ◽  
Nmda Channel

Spike-timing-dependent plasticity (STDP) is described by long-term potentiation (LTP), when a presynaptic event precedes a postsynaptic event, and by long-term depression (LTD), when the temporal order is reversed. In this article, we present a biophysical model of STDP based on a differential Hebbian learning rule (ISO learning). This rule correlates presynaptically the NMDA channel conductance with the derivative of the membrane potential at the synapse as the postsynaptic signal. The model is able to reproduce the generic STDP weight change characteristic. We find that (1) The actual shape of the weight change curve strongly depends on the NMDA channel characteristics and on the shape of the membrane potential at the synapse. (2) The typical antisymmetrical STDP curve (LTD and LTP) can become similar to a standard Hebbian characteristic (LTP only) without having to change the learning rule. This occurs if the membrane depolarization has a shallow onset and is long lasting. (3) It is known that the membrane potential varies along the dendrite as a result of the active or passive backpropagation of somatic spikes or because of local dendritic processes. As a consequence, our model predicts that learning properties will be different at different locations on the dendritic tree. In conclusion, such site-specific synaptic plasticity would provide a neuron with powerful learning capabilities.

A Cross-Correlated Delay Shift Supervised Learning Method for Spiking Neurons with Application to Interictal Spike Detection in Epilepsy

2017 ◽  
Vol 27 (03) ◽  
pp. 1750002 ◽  
Author(s):  
Lilin Guo ◽  
Zhenzhong Wang ◽  
Mercedes Cabrerizo ◽  
Malek Adjouadi
Keyword(s):  
Adaptive Learning ◽  
Learning Algorithm ◽  
Recognition Performance ◽  
Learning Rule ◽  
Classification Performance ◽  
Spiking Neurons ◽  
Learning Performance ◽  
Learning Method ◽  
Neuron Models ◽  
Interictal Spike

This study introduces a novel learning algorithm for spiking neurons, called CCDS, which is able to learn and reproduce arbitrary spike patterns in a supervised fashion allowing the processing of spatiotemporal information encoded in the precise timing of spikes. Unlike the Remote Supervised Method (ReSuMe), synapse delays and axonal delays in CCDS are variants which are modulated together with weights during learning. The CCDS rule is both biologically plausible and computationally efficient. The properties of this learning rule are investigated extensively through experimental evaluations in terms of reliability, adaptive learning performance, generality to different neuron models, learning in the presence of noise, effects of its learning parameters and classification performance. Results presented show that the CCDS learning method achieves learning accuracy and learning speed comparable with ReSuMe, but improves classification accuracy when compared to both the Spike Pattern Association Neuron (SPAN) learning rule and the Tempotron learning rule. The merit of CCDS rule is further validated on a practical example involving the automated detection of interictal spikes in EEG records of patients with epilepsy. Results again show that with proper encoding, the CCDS rule achieves good recognition performance.

Sign in / Sign up

Export Citation Format

Share Document