scholarly journals The dynamic neural code of the retina for natural scenes

2018 ◽  
Author(s):  
Niru Maheswaranathan ◽  
Lane T. McIntosh ◽  
Hidenori Tanaka ◽  
Satchel Grant ◽  
David B. Kastner ◽  
...  
Keyword(s):  
Visual Processing ◽  
Ganglion Cells ◽  
Predictive Coding ◽  
Neural Code ◽  
Natural Scenes ◽  
Large Set ◽  
Fundamental Limits ◽  
Fundamental Goal ◽  
Latent Effects ◽  
Highly Correlated

AbstractUnderstanding how the visual system encodes natural scenes is a fundamental goal of sensory neuroscience. We show here that a three-layer network model predicts the retinal response to natural scenes with an accuracy nearing the fundamental limits of predictability. The model’s internal structure is interpretable, in that model units are highly correlated with interneurons recorded separately and not used to fit the model. We further show the ethological relevance to natural visual processing of a diverse set of phenomena of complex motion encoding, adaptation and predictive coding. Our analysis uncovers a fast timescale of visual processing that is inaccessible directly from experimental data, showing unexpectedly that ganglion cells signal in distinct modes by rapidly (< 0.1 s) switching their selectivity for direction of motion, orientation, location and the sign of intensity. A new approach that decomposes ganglion cell responses into the contribution of interneurons reveals how the latent effects of parallel retinal circuits generate the response to any possible stimulus. These results reveal extremely flexible and rapid dynamics of the retinal code for natural visual stimuli, explaining the need for a large set of interneuron pathways to generate the dynamic neural code for natural scenes.

scholarly journals Heterogeneous Response Dynamics in Retinal Ganglion Cells: The Interplay of Predictive Coding and Adaptation

2010 ◽  
Vol 103 (6) ◽  
pp. 3184-3194 ◽  
Author(s):  
Sheila Nirenberg ◽  
Illya Bomash ◽  
Jonathan W. Pillow ◽  
Jonathan D. Victor
Keyword(s):  
White Noise ◽  
Visual Processing ◽  
Ganglion Cells ◽  
Predictive Coding ◽  
Sensory Systems ◽  
Natural Environments ◽  
Retinal Ganglion ◽  
Response Dynamics ◽  
Low Frequencies ◽  
Environmental Correlations

To make efficient use of their limited signaling capacity, sensory systems often use predictive coding. Predictive coding works by exploiting the statistical regularities of the environment—specifically, by filtering the sensory input to remove its predictable elements, thus enabling the neural signal to focus on what cannot be guessed. To do this, the neural filters must remove the environmental correlations. If predictive coding is to work well in multiple environments, sensory systems must adapt their filtering properties to fit each environment's statistics. Using the visual system as a model, we determine whether this happens. We compare retinal ganglion cell dynamics in two very different environments: white noise and natural. Because natural environments have more power than that of white noise at low temporal frequencies, predictive coding is expected to produce a suppression of low frequencies and an enhancement of high frequencies, compared with the behavior in a white-noise environment. We find that this holds, but only in part. First, predictive coding behavior is not uniform: most on cells manifest it, whereas off cells, on average, do not. Overlaid on this nonuniformity between cell classes is further nonuniformity within both cell classes. These findings indicate that functional considerations beyond predictive coding play an important role in shaping the dynamics of sensory adaptation. Moreover, the differences in behavior between on and off cell classes add to the growing evidence that these classes are not merely homogeneous mirror images of each other and suggest that their roles in visual processing are more complex than expected from the classic view.

scholarly journals Testing pseudo-linear models of responses to natural scenes in primate retina

2016 ◽  
Author(s):  
Alexander Heitman ◽  
Nora Brackbill ◽  
Martin Greschner ◽  
Alexander Sher ◽  
Alan M. Litke ◽  
...  
Keyword(s):  
Linear Model ◽  
Visual Processing ◽  
Large Scale ◽  
Linear Models ◽  
Ganglion Cells ◽  
Visual Stimuli ◽  
Natural Scenes ◽  
Natural Environments ◽  
Linear Filtering ◽  
Fixational Eye Movements

A central goal of systems neuroscience is to develop accurate quantitative models of how neural circuits process information. Prevalent models of light response in retinal ganglion cells (RGCs) usually begin with linear filtering over space and time, which reduces the high-dimensional visual stimulus to a simpler and more tractable scalar function of time that in turn determines the model output. Although these pseudo-linear models can accurately replicate RGC responses to stochastic stimuli, it is unclear whether the strong linearity assumption captures the function of the retina in the natural environment. This paper tests how accurately one pseudo-linear model, the generalized linear model (GLM), explains the responses of primate RGCs to naturalistic visual stimuli. Light responses from macaque RGCs were obtained using large-scale multi-electrode recordings, and two major cell types, ON and OFF parasol, were examined. Visual stimuli consisted of images of natural environments with simulated saccadic and fixational eye movements. The GLM accurately reproduced RGC responses to white noise stimuli, as observed previously, but did not generalize to predict RGC responses to naturalistic stimuli. It also failed to capture RGC responses when fitted and tested with naturalistic stimuli alone. Fitted scalar nonlinearities before and after the linear filtering stage were insufficient to correct the failures. These findings suggest that retinal signaling under natural conditions cannot be captured by models that begin with linear filtering, and emphasize the importance of additional spatial nonlinearities, gain control, and/or peripheral effects in the first stage of visual processing.

scholarly journals The emergence of multiple retinal cell types through efficient coding of natural movies

2018 ◽  
Author(s):  
Samuel A. Ocko ◽  
Jack Lindsey ◽  
Surya Ganguli ◽  
Stephane Deny
Keyword(s):  
Visual Processing ◽  
Visual Information ◽  
Ganglion Cells ◽  
Receptive Fields ◽  
Cell Types ◽  
Retinal Cell ◽  
Retinal Eccentricity ◽  
Natural Scenes ◽  
Efficient Coding ◽  
Cell Densities

AbstractOne of the most striking aspects of early visual processing in the retina is the immediate parcellation of visual information into multiple parallel pathways, formed by different retinal ganglion cell types each tiling the entire visual field. Existing theories of efficient coding have been unable to account for the functional advantages of such cell-type diversity in encoding natural scenes. Here we go beyond previous theories to analyze how a simple linear retinal encoding model with different convolutional cell types efficiently encodes naturalistic spatiotemporal movies given a fixed firing rate budget. We find that optimizing the receptive fields and cell densities of two cell types makes them match the properties of the two main cell types in the primate retina, midget and parasol cells, in terms of spatial and temporal sensitivity, cell spacing, and their relative ratio. Moreover, our theory gives a precise account of how the ratio of midget to parasol cells decreases with retinal eccentricity. Also, we train a nonlinear encoding model with a rectifying nonlinearity to efficiently encode naturalistic movies, and again find emergent receptive fields resembling those of midget and parasol cells that are now further subdivided into ON and OFF types. Thus our work provides a theoretical justification, based on the efficient coding of natural movies, for the existence of the four most dominant cell types in the primate retina that together comprise 70% of all ganglion cells.

scholarly journals Pathway-specific asymmetries between ON and OFF visual signals

2018 ◽  
Author(s):  
Sneha Ravi ◽  
Daniel Ahn ◽  
Martin Greschner ◽  
E.J Chichilnisky ◽  
Greg D. Field
Keyword(s):  
Visual Processing ◽  
Functional Organization ◽  
Ganglion Cells ◽  
Temporal Integration ◽  
Receptive Fields ◽  
Visual Signals ◽  
Natural Scenes ◽  
Alpha Cells ◽  
Spatial Integration ◽  
On And Off Pathways

AbstractVisual processing is largely organized into ON and OFF pathways that signal stimulus increments and decrements, respectively. These pathways exhibit natural pairings based on morphological and physiological similarities, such as ON and OFF alpha ganglion cells in the mammalian retina. Several studies have noted asymmetries in the properties of ON and OFF pathways. For example, the spatial receptive fields (RFs) of OFF alpha cells are systematically smaller than ON alpha cells. Analysis of natural scenes suggests these asymmetries are optimal for visual encoding. To test the generality of ON-OFF asymmetries, we measured the spatiotemporal RF properties of multiple RGC types in rat retina. Through a quantitative and serial classification, we identified three functional pairs of ON and OFF RGCs. We analyzed the structure of their RFs and compared spatial integration, temporal integration, and gain across ON and OFF pairs. Similar to previous results from cat and primate, RGC types with larger spatial RFs exhibited briefer temporal integration and higher gain. However, each pair of ON and OFF RGC types exhibited distinct asymmetric relationships between receptive field properties, some of which were opposite to previous reports. These results reveal the functional organization of six RGC types in the rodent retina and indicate that ON-OFF asymmetries are pathway specific.Significance StatementCircuits that process sensory input frequently process increments separately from decrements, so called ‘ON’ and ‘OFF’ responses. Theoretical studies indicate this separation, and associated asymmetries in ON and OFF pathways, may be beneficial for encoding natural stimuli. However, the generality of ON and OFF pathway asymmetries has not been tested. Here we compare the functional properties of three distinct pairs of ON and OFF pathways in the rodent retina and show their asymmetries are pathway specific. These results provide a new view on the partitioning of vision across diverse ON and OFF signaling pathways

scholarly journals Summation of perceptual cues in natural visual scenes

2008 ◽  
Vol 275 (1649) ◽  
pp. 2299-2308 ◽  
Author(s):  
M To ◽  
P.G Lovell ◽  
T Troscianko ◽  
D.J Tolhurst
Keyword(s):  
Visual Processing ◽  
Neural System ◽  
Stimulus Dimension ◽  
Two Dimensions ◽  
Natural Scenes ◽  
Perceptual Cues ◽  
Combination Rules ◽  
Threshold Detection ◽  
Visual Scenes ◽  
Image Pairs

Natural visual scenes are rich in information, and any neural system analysing them must piece together the many messages from large arrays of diverse feature detectors. It is known how threshold detection of compound visual stimuli (sinusoidal gratings) is determined by their components' thresholds. We investigate whether similar combination rules apply to the perception of the complex and suprathreshold visual elements in naturalistic visual images. Observers gave magnitude estimations (ratings) of the perceived differences between pairs of images made from photographs of natural scenes. Images in some pairs differed along one stimulus dimension such as object colour, location, size or blur. But, for other image pairs, there were composite differences along two dimensions (e.g. both colour and object-location might change). We examined whether the ratings for such composite pairs could be predicted from the two ratings for the respective pairs in which only one stimulus dimension had changed. We found a pooling relationship similar to that proposed for simple stimuli: Minkowski summation with exponent 2.84 yielded the best predictive power ( r =0.96), an exponent similar to that generally reported for compound grating detection. This suggests that theories based on detecting simple stimuli can encompass visual processing of complex, suprathreshold stimuli.

Independence of color and luminance edges in natural scenes

2009 ◽  
Vol 26 (1) ◽  
pp. 35-49 ◽  
Author(s):  
THORSTEN HANSEN ◽  
KARL R. GEGENFURTNER
Keyword(s):  
Visual Processing ◽  
Artificial Vision ◽  
Natural Scenes ◽  
Natural Scene ◽  
Natural Scene Statistics ◽  
Form Vision ◽  
Independent Information ◽  
Edge Contrast ◽  
Source Of Information ◽  
Gain Access

AbstractForm vision is traditionally regarded as processing primarily achromatic information. Previous investigations into the statistics of color and luminance in natural scenes have claimed that luminance and chromatic edges are not independent of each other and that any chromatic edge most likely occurs together with a luminance edge of similar strength. Here we computed the joint statistics of luminance and chromatic edges in over 700 calibrated color images from natural scenes. We found that isoluminant edges exist in natural scenes and were not rarer than pure luminance edges. Most edges combined luminance and chromatic information but to varying degrees such that luminance and chromatic edges were statistically independent of each other. Independence increased along successive stages of visual processing from cones via postreceptoral color-opponent channels to edges. The results show that chromatic edge contrast is an independent source of information that can be linearly combined with other cues for the proper segmentation of objects in natural and artificial vision systems. Color vision may have evolved in response to the natural scene statistics to gain access to this independent information.

scholarly journals Depth in convolutional neural networks solves scene segmentation

2019 ◽  
Author(s):  
N Seijdel ◽  
N Tsakmakidis ◽  
EHF De Haan ◽  
SM Bohte ◽  
HS Scholte
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Visual Processing ◽  
Human Performance ◽  
Object Identification ◽  
Image Features ◽  
Background Information ◽  
Natural Scenes ◽  
Scene Segmentation ◽  
Deep Convolutional Neural Networks

AbstractFeedforward deep convolutional neural networks (DCNNs) are, under specific conditions, matching and even surpassing human performance in object recognition in natural scenes. This performance suggests that the analysis of a loose collection of image features could support the recognition of natural object categories, without dedicated systems to solve specific visual subtasks. Research in humans however suggests that while feedforward activity may suffice for sparse scenes with isolated objects, additional visual operations (‘routines’) that aid the recognition process (e.g. segmentation or grouping) are needed for more complex scenes. Linking human visual processing to performance of DCNNs with increasing depth, we here explored if, how, and when object information is differentiated from the backgrounds they appear on. To this end, we controlled the information in both objects and backgrounds, as well as the relationship between them by adding noise, manipulating background congruence and systematically occluding parts of the image. Results indicate that with an increase in network depth, there is an increase in the distinction between object- and background information. For more shallow networks, results indicated a benefit of training on segmented objects. Overall, these results indicate that, de facto, scene segmentation can be performed by a network of sufficient depth. We conclude that the human brain could perform scene segmentation in the context of object identification without an explicit mechanism, by selecting or “binding” features that belong to the object and ignoring other features, in a manner similar to a very deep convolutional neural network.

scholarly journals Prediction-error signals to violated expectations about person identity and head orientation are doubly-dissociated across the dorsal and ventral visual streams

2018 ◽  
Author(s):  
Jonathan E. Robinson ◽  
Will Woods ◽  
Sumie Leung ◽  
Jordy Kaufman ◽  
Michael Breakspear ◽  
...  
Keyword(s):  
Error Detection ◽  
Visual Processing ◽  
Prediction Error ◽  
Predictive Coding ◽  
Angular Gyrus ◽  
Head Orientation ◽  
Visual Stream ◽  
Person Identity ◽  
Violated Expectations ◽  
Double Dissociations

AbstractPredictive coding theories of perception suggest the importance of constantly updated internal models of the world in predicting future sensory inputs. One implication of such models is that cortical regions whose function is to resolve particular stimulus attributes should also signal prediction violations with respect to those same stimulus attributes. Previously, through carefully designed experiments, we have demonstrated early-mid latency EEG/MEG prediction-error signals in the dorsal visual stream to violated expectations about stimulus orientation/trajectory, with localisations consistent with cortical areas processing motion and orientation. Here we extend those methods to simultaneously investigate the predictive processes in both dorsal and ventral visual streams. In this MEG study we employed a contextual trajectory paradigm that builds expectations using a series of image presentations. We created expectations about both face orientation and identity, either of which can subsequently be violated. Crucially this paradigm allows us to parametrically test double dissociations between these different types of violations. The study identified double dissociations across the type of violation in the dorsal and ventral visual streams, such that the right fusiform gyrus showed greater evidence of prediction-error signals to Identity violations than to Orientation violations, whereas the left angular gyrus and postcentral gyrus showed the opposite pattern of results. Our results suggest comparable processes for error checking and context updating in high-level expectations instantiated across both perceptual streams. Perceptual prediction-error signalling is initiated in regions associated with the processing of different stimulus properties.Significance StatementVisual processing occurs along ‘what’ and ‘where’ information streams that run, respectively along the ventral and dorsal surface of the posterior brain. Predictive coding models of perception imply prediction-error detection processes that are instantiated at the level where particular stimulus attributes are parsed. This implies that, for instance, when considering face stimuli, signals arising through violated expectations about the person identity of the stimulus should localise to the ventral stream, whereas signals arising through violated expectations about head orientation should localise to the dorsal stream. We test this in a magnetoencephalography source localisation study. The analysis confirmed that prediction-error signals to identity versus head-orientation occur with similar latency, but activate doubly-dissociated brain regions along ventral and dorsal processing streams.

scholarly journals POU domain factor Brn-3b is required for the development of a large set of retinal ganglion cells.

1996 ◽  
Vol 93 (9) ◽  
pp. 3920-3925 ◽  
Author(s):  
L. Gan ◽  
M. Xiang ◽  
L. Zhou ◽  
D. S. Wagner ◽  
W. H. Klein ◽  
...  
Keyword(s):  
Retinal Ganglion Cells ◽  
Ganglion Cells ◽  
Large Set ◽  
Retinal Ganglion ◽  
Pou Domain
Sign in / Sign up

Export Citation Format

Share Document