scholarly journals Towards a unified theory of efficient, predictive and sparse coding

2017 ◽  
Author(s):  
Matthew Chalk ◽  
Olivier Marre ◽  
Gašper Tkačik
Keyword(s):  
Sparse Coding ◽  
Visual Motion ◽  
Synergistic Effects ◽  
Predictive Coding ◽  
Low Noise ◽  
Neural Responses ◽  
Unified Framework ◽  
Response Properties ◽  
Efficient Coding ◽  
Special Cases

A central goal in theoretical neuroscience is to predict the response properties of sensory neurons from first principles. Several theories have been proposed to this end. “Efficient coding” posits that neural circuits maximise information encoded about their inputs. “Sparse coding” posits that individual neurons respond selectively to specific, rarely occurring, features. Finally, “predictive coding” posits that neurons preferentially encode stimuli that are useful for making predictions. Except in special cases, it is unclear how these theories relate to each other, or what is expected if different coding objectives are combined. To address this question, we developed a unified framework that encompasses these previous theories and extends to new regimes, such as sparse predictive coding. We explore cases when different coding objectives exert conflicting or synergistic effects on neural response properties. We show that predictive coding can lead neurons to either correlate or decorrelate their inputs, depending on presented stimuli, while (at low-noise) efficient coding always predicts decorrelation. We compare predictive versus sparse coding of natural movies, showing that the two theories predict qualitatively different neural responses to visual motion. Our approach promises a way to explain the observed diversity of sensory neural responses, as due to a multiplicity of functional goals performed by different cell types and/or circuits.

scholarly journals Toward a unified theory of efficient, predictive, and sparse coding

2017 ◽  
Vol 115 (1) ◽  
pp. 186-191 ◽  
Author(s):  
Matthew Chalk ◽  
Olivier Marre ◽  
Gašper Tkačik
Keyword(s):  
Sensory Neurons ◽  
Sparse Coding ◽  
Visual Motion ◽  
Predictive Coding ◽  
Cell Types ◽  
Low Noise ◽  
Neural Responses ◽  
Unified Framework ◽  
Efficient Coding ◽  
The Past

A central goal in theoretical neuroscience is to predict the response properties of sensory neurons from first principles. To this end, “efficient coding” posits that sensory neurons encode maximal information about their inputs given internal constraints. There exist, however, many variants of efficient coding (e.g., redundancy reduction, different formulations of predictive coding, robust coding, sparse coding, etc.), differing in their regimes of applicability, in the relevance of signals to be encoded, and in the choice of constraints. It is unclear how these types of efficient coding relate or what is expected when different coding objectives are combined. Here we present a unified framework that encompasses previously proposed efficient coding models and extends to unique regimes. We show that optimizing neural responses to encode predictive information can lead them to either correlate or decorrelate their inputs, depending on the stimulus statistics; in contrast, at low noise, efficiently encoding the past always predicts decorrelation. Later, we investigate coding of naturalistic movies and show that qualitatively different types of visual motion tuning and levels of response sparsity are predicted, depending on whether the objective is to recover the past or predict the future. Our approach promises a way to explain the observed diversity of sensory neural responses, as due to multiple functional goals and constraints fulfilled by different cell types and/or circuits.

scholarly journals Spatio-temporally efficient coding assigns functions to hierarchical structures of the visual system

2021 ◽  
Author(s):  
Duho Sihn ◽  
Sung-Phil Kim
Keyword(s):  
Visual System ◽  
Predictive Coding ◽  
Hierarchical Structures ◽  
Brain Structures ◽  
External World ◽  
Neural Responses ◽  
Brain Areas ◽  
Efficient Coding ◽  
Neural Representations ◽  
Computational Processes

Hierarchical structures constitute a wide array of brain areas, including the visual system. One of the important questions regarding visual hierarchical structures is to identify computational principles for assigning functions that represent the external world to hierarchical structures of the visual system. Given that visual hierarchical structures contain both bottom-up and top-down pathways, the derived principles should encompass these bidirectional pathways. However, existing principles such as predictive coding do not provide an effective principle for bidirectional pathways. Therefore, we propose a novel computational principle for visual hierarchical structures as spatio-temporally efficient coding underscored by the efficient use of given resources in both neural activity space and processing time. This coding principle optimises bidirectional predictions over hierarchical structures by simultaneously minimising temporally differences in neural responses and maximising entropy in neural representations. Simulations demonstrated that the proposed spatio-temporally efficient coding was able to assign the function of appropriate neural representations of natural visual scenes to visual hierarchical structures. Furthermore, spatio-temporally efficient coding was able to predict well-known phenomena, including deviations in neural responses to unfamiliar inputs and bias in preferred orientations. Our proposed spatio-temporally efficient coding may facilitate deeper mechanistic understanding of the computational processes of hierarchical brain structures.

scholarly journals Interpreting Neuronal Population Activity by Reconstruction: Unified Framework With Application to Hippocampal Place Cells

1998 ◽  
Vol 79 (2) ◽  
pp. 1017-1044 ◽  
Author(s):  
Kechen Zhang ◽  
Iris Ginzburg ◽  
Bruce L. McNaughton ◽  
Terrence J. Sejnowski
Keyword(s):  
Bayesian Methods ◽  
Neuronal Population ◽  
Place Cells ◽  
The Body ◽  
Unified Framework ◽  
Population Activity ◽  
Reconstruction Methods ◽  
Special Cases ◽  
Physical Variables ◽  
The Brain

Zhang, Kechen, Iris Ginzburg, Bruce L. McNaughton, and Terrence J. Sejnowski. Interpreting neuronal population activity by reconstruction: unified framework with application to hippocampal place cells. J. Neurophysiol. 79: 1017–1044, 1998. Physical variables such as the orientation of a line in the visual field or the location of the body in space are coded as activity levels in populations of neurons. Reconstruction or decoding is an inverse problem in which the physical variables are estimated from observed neural activity. Reconstruction is useful first in quantifying how much information about the physical variables is present in the population and, second, in providing insight into how the brain might use distributed representations in solving related computational problems such as visual object recognition and spatial navigation. Two classes of reconstruction methods, namely, probabilistic or Bayesian methods and basis function methods, are discussed. They include important existing methods as special cases, such as population vector coding, optimal linear estimation, and template matching. As a representative example for the reconstruction problem, different methods were applied to multi-electrode spike train data from hippocampal place cells in freely moving rats. The reconstruction accuracy of the trajectories of the rats was compared for the different methods. Bayesian methods were especially accurate when a continuity constraint was enforced, and the best errors were within a factor of two of the information-theoretic limit on how accurate any reconstruction can be and were comparable with the intrinsic experimental errors in position tracking. In addition, the reconstruction analysis uncovered some interesting aspects of place cell activity, such as the tendency for erratic jumps of the reconstructed trajectory when the animal stopped running. In general, the theoretical values of the minimal achievable reconstruction errors quantify how accurately a physical variable is encoded in the neuronal population in the sense of mean square error, regardless of the method used for reading out the information. One related result is that the theoretical accuracy is independent of the width of the Gaussian tuning function only in two dimensions. Finally, all the reconstruction methods considered in this paper can be implemented by a unified neural network architecture, which the brain feasibly could use to solve related problems.

Rural and Urban Population Changes and the Stages of Economic Development: A Unified Approach

1983 ◽  
Vol 15 (9) ◽  
pp. 1161-1174 ◽  
Author(s):  
T Miyao
Keyword(s):  
Economic Development ◽  
Urban Areas ◽  
Unified Framework ◽  
Rural And Urban Areas ◽  
Rural And Urban ◽  
Special Cases ◽  
Advanced Stages ◽  
Dynamic Path ◽  
Stages Of Economic Development ◽  
Shed Light

A model is presented which can yield a dynamic path of economic development from very early stages through more advanced stages in an attempt to shed light on what Alonso called the “five bell-shapes” within a unified framework. The model is able to explain not only the occurrence of a downturn in the rural population after the initial phase of population growth both in rural and urban areas, but also the delayed occurrence of such a downturn in many present-day developing countries. The author then focuses the later stages of economic development and explains two alternative courses of urbanization, namely, the reversal process and the continual-growth process, as special cases of the general model; which of the courses occurs depends on the value of the elasticity of urban agglomeration-economies.

Placebos in chronic pain: evidence, theory, ethics, and use in clinical practice

BMJ ◽  
2020 ◽  
pp. m1668 ◽  
Author(s):  
Ted J Kaptchuk ◽  
Christopher C Hemond ◽  
Franklin G Miller
Keyword(s):  
Chronic Pain ◽  
Clinical Practice ◽  
Predictive Coding ◽  
Evidence Theory ◽  
Placebo Treatment ◽  
Placebo Effects ◽  
Unified Framework ◽  
Open Label ◽  
Double Blind ◽  
Bayesian Brain

ABSTRACTDespite their ubiquitous presence, placebos and placebo effects retain an ambiguous and unsettling presence in biomedicine. Specifically focused on chronic pain, this review examines the effect of placebo treatment under three distinct frameworks: double blind, deception, and open label honestly prescribed. These specific conditions do not necessarily differentially modify placebo outcomes. Psychological, clinical, and neurological theories of placebo effects are scrutinized. In chronic pain, conscious expectation does not reliably predict placebo effects. A supportive patient-physician relationship may enhance placebo effects. This review highlights “predictive coding” and “bayesian brain” as emerging models derived from computational neurobiology that offer a unified framework to explain the heterogeneous evidence on placebos. These models invert the dogma of the brain as a stimulus driven organ to one in which perception relies heavily on learnt, top down, cortical predictions to infer the source of incoming sensory data. In predictive coding/bayesian brain, both chronic pain (significantly modulated by central sensitization) and its alleviation with placebo treatment are explicated as centrally encoded, mostly non-conscious, bayesian biases. The review then evaluates seven ways in which placebos are used in clinical practice and research and their bioethical implications. In this way, it shows that placebo effects are evidence based, clinically relevant, and potentially ethical tools for relieving chronic pain.

scholarly journals Is predictability salient? A study of attentional capture by auditory patterns

2017 ◽  
Vol 372 (1714) ◽  
pp. 20160105 ◽  
Author(s):  
Rosy Southwell ◽  
Anna Baumann ◽  
Cécile Gal ◽  
Nicolas Barascud ◽  
Karl Friston ◽  
...  
Keyword(s):  
Attentional Capture ◽  
Predictive Coding ◽  
Visual Scene ◽  
Scene Analysis ◽  
Neural Responses ◽  
Eeg Data ◽  
Visual Domain ◽  
The Brain ◽  
Repeating Patterns

In this series of behavioural and electroencephalography (EEG) experiments, we investigate the extent to which repeating patterns of sounds capture attention. Work in the visual domain has revealed attentional capture by statistically predictable stimuli, consistent with predictive coding accounts which suggest that attention is drawn to sensory regularities. Here, stimuli comprised rapid sequences of tone pips, arranged in regular (REG) or random (RAND) patterns. EEG data demonstrate that the brain rapidly recognizes predictable patterns manifested as a rapid increase in responses to REG relative to RAND sequences. This increase is reminiscent of the increase in gain on neural responses to attended stimuli often seen in the neuroimaging literature, and thus consistent with the hypothesis that predictable sequences draw attention. To study potential attentional capture by auditory regularities, we used REG and RAND sequences in two different behavioural tasks designed to reveal effects of attentional capture by regularity. Overall, the pattern of results suggests that regularity does not capture attention. This article is part of the themed issue ‘Auditory and visual scene analysis’.

Sign in / Sign up

Export Citation Format

Share Document