Low-level properties of natural images predict topographic patterns of neural response in the ventral visual pathway

AbstractHumans recognize individual faces regardless of variation in the facial view. The view-tuned face neurons in the inferior temporal (IT) cortex are regarded as the neural substrate for view-invariant face recognition. This study approximated visual features encoded by these neurons as combinations of local orientations and colors, originated from natural image fragments. The resultant features reproduced the preference of these neurons to particular facial views. We also found that faces of one identity were separable from the faces of other identities in a space where each axis represented one of these features. These results suggested that view-invariant face representation was established by combining view sensitive visual features. The face representation with these features suggested that, with respect to view-invariant face representation, the seemingly complex and deeply layered ventral visual pathway can be approximated via a shallow network, comprised of layers of low-level processing for local orientations and colors (V1/V2-level) and the layers which detect particular sets of low-level elements derived from natural image fragments (IT-level).

Download Full-text

Reconstructing feedback representations in ventral visual pathway with a generative adversarial autoencoder

10.1101/2020.07.23.218859 ◽

2020 ◽

Author(s):

Haider Al-Tahan ◽

Yalda Mohsenzadeh

Keyword(s):

Neural Network ◽

Visual Information ◽

Visual Pathway ◽

Functional Magnetic Resonance ◽

Low Level ◽

Ventral Visual Pathway ◽

High Level ◽

Feedback Connections ◽

The Brain ◽

Insight Into

AbstractWhile vision evokes a dense network of feedforward and feedback neural processes in the brain, visual processes are primarily modeled with feedforward hierarchical neural networks, leaving the computational role of feedback processes poorly understood. Here, we developed a generative autoencoder neural network model and adversarially trained it on a categorically diverse data set of images. We hypothesized that the feedback processes in the ventral visual pathway can be represented by reconstruction of the visual information performed by the generative model. We compared representational similarity of the activity patterns in the proposed model with temporal (magnetoencephalography) and spatial (functional magnetic resonance imaging) visual brain responses. The proposed generative model identified two segregated neural dynamics in the visual brain. A temporal hierarchy of processes transforming low level visual information into high level semantics in the feedforward sweep, and a temporally later dynamics of inverse processes reconstructing low level visual information from a high level latent representation in the feedback sweep. Our results append to previous studies on neural feedback processes by presenting a new insight into the algorithmic function and the information carried by the feedback processes in the ventral visual pathway.Author summaryIt has been shown that the ventral visual cortex consists of a dense network of regions with feedforward and feedback connections. The feedforward path processes visual inputs along a hierarchy of cortical areas that starts in early visual cortex (an area tuned to low level features e.g. edges/corners) and ends in inferior temporal cortex (an area that responds to higher level categorical contents e.g. faces/objects). Alternatively, the feedback connections modulate neuronal responses in this hierarchy by broadcasting information from higher to lower areas. In recent years, deep neural network models which are trained on object recognition tasks achieved human-level performance and showed similar activation patterns to the visual brain. In this work, we developed a generative neural network model that consists of encoding and decoding sub-networks. By comparing this computational model with the human brain temporal (magnetoencephalography) and spatial (functional magnetic resonance imaging) response patterns, we found that the encoder processes resemble the brain feedforward processing dynamics and the decoder shares similarity with the brain feedback processing dynamics. These results provide an algorithmic insight into the spatiotemporal dynamics of feedforward and feedback processes in biological vision.

Download Full-text

Category-selective patterns of neural response to scrambled images in the ventral visual pathway.

Journal of Vision ◽

10.1167/15.12.622 ◽

2015 ◽

Vol 15 (12) ◽

pp. 622 ◽

Cited By ~ 1

Author(s):

David Coggan ◽

Wanling Liu ◽

Daniel Baker ◽

Timothy Andrews

Keyword(s):

Visual Pathway ◽

Neural Response ◽

Ventral Visual Pathway

Download Full-text

Oscillatory Dynamics of Perceptual to Conceptual Transformations in the Ventral Visual Pathway

Journal of Cognitive Neuroscience ◽

10.1162/jocn_a_01325 ◽

2018 ◽

Vol 30 (11) ◽

pp. 1590-1605 ◽

Cited By ~ 7

Author(s):

Alex Clarke ◽

Barry J. Devereux ◽

Lorraine K. Tyler

Keyword(s):

Visual Information ◽

Computational Models ◽

Semantic Information ◽

Low Frequency ◽

Visual Pathway ◽

Semantic Representations ◽

Low Level ◽

Oscillatory Dynamics ◽

Incremental Model ◽

Ventral Visual Pathway

Object recognition requires dynamic transformations of low-level visual inputs to complex semantic representations. Although this process depends on the ventral visual pathway, we lack an incremental account from low-level inputs to semantic representations and the mechanistic details of these dynamics. Here we combine computational models of vision with semantics and test the output of the incremental model against patterns of neural oscillations recorded with magnetoencephalography in humans. Representational similarity analysis showed visual information was represented in low-frequency activity throughout the ventral visual pathway, and semantic information was represented in theta activity. Furthermore, directed connectivity showed visual information travels through feedforward connections, whereas visual information is transformed into semantic representations through feedforward and feedback activity, centered on the anterior temporal lobe. Our research highlights that the complex transformations between visual and semantic information is driven by feedforward and recurrent dynamics resulting in object-specific semantics.

Download Full-text

Spatial properties of objects predict patterns of neural response in the ventral visual pathway

NeuroImage ◽

10.1016/j.neuroimage.2015.11.043 ◽

2016 ◽

Vol 126 ◽

pp. 173-183 ◽

Cited By ~ 15

Author(s):

David M. Watson ◽

Andrew W. Young ◽

Timothy J. Andrews

Keyword(s):

Visual Pathway ◽

Neural Response ◽

Spatial Properties ◽

Ventral Visual Pathway

Download Full-text

Oscillatory dynamics of perceptual to conceptual transformations in the ventral visual pathway

10.1101/259127 ◽

2018 ◽

Author(s):

Alex Clarke ◽

Barry J. Devereux ◽

Lorraine K. Tyler

Keyword(s):

Visual Information ◽

Computational Models ◽

Semantic Information ◽

Visual Pathway ◽

Semantic Representations ◽

Low Level ◽

Oscillatory Dynamics ◽

Incremental Model ◽

Visual Inputs ◽

Ventral Visual Pathway

AbstractObject recognition requires dynamic transformations of low-level visual inputs to complex semantic representations. While this process depends on the ventral visual pathway (VVP), we lack an incremental account from low-level inputs to semantic representations, and the mechanistic details of these dynamics. Here we combine computational models of vision with semantics, and test the output of the incremental model against patterns of neural oscillations recorded with MEG in humans. Representational Similarity Analysis showed visual information was represented in alpha activity throughout the VVP, and semantic information was represented in theta activity. Furthermore, informational connectivity showed visual information travels through feedforward connections, while visual information is transformed into semantic representations through feedforward and feedback activity, centered on the anterior temporal lobe. Our research highlights that the complex transformations between visual and semantic information is driven by feedforward and recurrent dynamics resulting in object-specific semantics.

Download Full-text