BEAN: Interpretable and Efficient Learning With Biologically-Enhanced Artificial Neuronal Assembly Regularization

Deep neural networks (DNNs) are known for extracting useful information from large amounts of data. However, the representations learned in DNNs are typically hard to interpret, especially in dense layers. One crucial issue of the classical DNN model such as multilayer perceptron (MLP) is that neurons in the same layer of DNNs are conditionally independent of each other, which makes co-training and emergence of higher modularity difficult. In contrast to DNNs, biological neurons in mammalian brains display substantial dependency patterns. Specifically, biological neural networks encode representations by so-called neuronal assemblies: groups of neurons interconnected by strong synaptic interactions and sharing joint semantic content. The resulting population coding is essential for human cognitive and mnemonic processes. Here, we propose a novel Biologically Enhanced Artificial Neuronal assembly (BEAN) regularization1 to model neuronal correlations and dependencies, inspired by cell assembly theory from neuroscience. Experimental results show that BEAN enables the formation of interpretable neuronal functional clusters and consequently promotes a sparse, memory/computation-efficient network without loss of model performance. Moreover, our few-shot learning experiments demonstrate that BEAN could also enhance the generalizability of the model when training samples are extremely limited.

Suppressive control of optokinetic and vestibular nystagmus by the primate frontal eye field

In this study, electrical stimulation in the frontal eye field (FEF) suppressed the quick and slow phases of optokinetic and vestibular nystagmus at an intensity subthreshold for eliciting saccades. Furthermore, the activity of fixation neurons in the FEF was related to the suppression of optokinetic and vestibular nystagmus by visual fixation. This suggests that a common neuronal assembly in the FEF may contribute to the suppressive control of different functional classes of eye movements.

Neuronal Assembly

To effectively send a message, a single neuron must cooperate with its peers. Such cooperation can be achieved by synchronizing their spikes together within the time window limited by the ability of the downstream reader neuron to integrate the incoming signals. Therefore, the cell assembly, defined from the point of view of the reader neuron, can be considered as a unit of neuronal communication, a “neuronal letter.” Acting in assemblies has several advantages. A cooperative assembly partnership tolerates spike rate variation in individual cells effectively because the total excitatory effect of the assembly is what matters to the reader mechanism. Interacting assembly members can compute probabilities rather than convey deterministic information and can robustly tolerate noise even if the individual members respond probabilistically.

Internally Organized Activity During Offline Brain States

A prime example of internally organized patterns is observed during sleep. The best studied of these is the sharp wave ripple in the hippocampus. Neuronal sequences during ripple events reach back to the past to replay snippets of waking experience at times when the brain is disengaged from the outside world. This process may consolidate episodic memories and stitch together discontiguous experiences, thereby giving rise to creative thoughts. In addition, neuronal assembly sequences during ripples also act as internalized, vicarious, trial-and-error mechanisms that can assist with subconscious optimization of future plans. Because the same neuronal substrate can perform both retrospective and prospective operations, it is not clear whether the traditional separation of postdiction (i.e., memory) from prediction (i.e., planning) is justified.

Synapse formation: from cellular and molecular mechanisms to neurodevelopmental and neurodegenerative disorders

The precise patterns of neuronal assembly during development determine all functional outputs of a nervous system; these may range from simple reflexes to learning, memory, cognition, etc. To understand how brain functions and how best to repair it after injury, disease, or trauma, it is imperative that we first seek to define fundamental steps mediating this neuronal assembly. To acquire the sophisticated ensemble of highly specialized networks seen in a mature brain, all proliferated and migrated neurons must extend their axonal and dendritic processes toward targets, which are often located at some distance. Upon contact with potential partners, neurons must undergo dramatic structural changes to become either a pre- or a postsynaptic neuron. This connectivity is cemented through specialized structures termed synapses. Both structurally and functionally, the newly formed synapses are, however, not static as they undergo consistent changes in order for an animal to meet its behavioral needs in a changing environment. These changes may be either in the form of new synapses or an enhancement of their synaptic efficacy, referred to as synaptic plasticity. Thus, synapse formation is not restricted to neurodevelopment; it is a process that remains active throughout life. As the brain ages, either the lack of neuronal activity or cell death render synapses dysfunctional, thus giving rise to neurodegenerative disorders. This review seeks to highlight salient steps that are involved in a neuron’s journey, starting with the establishment, maturation, and consolidation of synapses; we particularly focus on identifying key players involved in the synaptogenic program. We hope that this endeavor will not only help the beginners in this field to understand how brain networks are assembled in the first place but also shed light on various neurodevelopmental, neurological, neurodegenerative, and neuropsychiatric disorders that involve synaptic inactivity or dysfunction.

Emergent cortical circuit dynamics contain dense, interwoven ensembles of spike sequences

Temporal codes are theoretically powerful encoding schemes, but their precise form in the neocortex remains unknown in part because of the large number of possible codes and the difficulty in disambiguating informative spikes from statistical noise. A biologically plausible and computationally powerful temporal coding scheme is the Hebbian assembly phase sequence (APS), which predicts reliable propagation of spikes between functionally related assemblies of neurons. Here, we sought to measure the inherent capacity of neocortical networks to produce reliable sequences of spikes, as would be predicted by an APS code. To record microcircuit activity, the scale at which computation is implemented, we used two-photon calcium imaging to densely sample spontaneous activity in murine neocortical networks ex vivo. We show that the population spike histogram is sufficient to produce a spatiotemporal progression of activity across the population. To more comprehensively evaluate the capacity for sequential spiking that cannot be explained by the overall population spiking, we identify statistically significant spike sequences. We found a large repertoire of sequence spikes that collectively comprise the majority of spiking in the circuit. Sequences manifest probabilistically and share neuron membership, resulting in unique ensembles of interwoven sequences characterizing individual spatiotemporal progressions of activity. Distillation of population dynamics into its constituent sequences provides a way to capture trial-to-trial variability and may prove to be a powerful decoding substrate in vivo. Informed by these data, we suggest that the Hebbian APS be reformulated as interwoven sequences with flexible assembly membership due to shared overlapping neurons. NEW & NOTEWORTHY Neocortical computation occurs largely within microcircuits comprised of individual neurons and their connections within small volumes (<500 μm3). We found evidence for a long-postulated temporal code, the Hebbian assembly phase sequence, by identifying repeated and co-occurring sequences of spikes. Variance in population activity across trials was explained in part by the ensemble of active sequences. The presence of interwoven sequences suggests that neuronal assembly structure can be variable and is determined by previous activity.