Optimal temporal resolution for decoding of visual stimuli in inferior temporal cortex

Image features computed by specific convolutional artificial neural networks (ANNs) can be used to make state-of-the-art predictions of primate ventral stream responses to visual stimuli.However, in addition to selecting the specific ANN and layer that is used, the modeler makes other choices in preprocessing the stimulus image and generating brain predictions from ANN features. The effect of these choices on brain predictivity is currently underexplored.Here, we directly evaluated many of these choices by performing a grid search over network architectures, layers, image preprocessing strategies, feature pooling mechanisms, and the use of dimensionality reduction. Our goal was to identify model configurations that produce responses to visual stimuli that are most similar to the human neural representations, as measured by human fMRI and MEG responses. In total, we evaluated more than 140,338 model configurations. We found that specific configurations of CORnet-S best predicted fMRI responses in early visual cortex, and CORnet-R and SqueezeNet models best predicted fMRI responses in inferior temporal cortex. We found specific configurations of VGG-16 and CORnet-S models that best predicted the MEG responses.We also observed that downsizing input images to ~50-75% of the input tensor size lead to better performing models compared to no downsizing (the default choice in most brain models for vision). Taken together, we present evidence that brain predictivity is sensitive not only to which ANN architecture and layer is used, but choices in image preprocessing and feature postprocessing, and these choices should be further explored.

Download Full-text

Differential modulation of neuronal activity in the macaque inferior temporal cortex by attention to global and local features of visual stimuli

Neuroscience Research ◽

10.1016/s0168-0102(00)81115-3 ◽

2000 ◽

Vol 38 ◽

pp. S44

Author(s):

H Tanaka

Keyword(s):

Neuronal Activity ◽

Visual Stimuli ◽

Temporal Cortex ◽

Local Features ◽

Inferior Temporal Cortex ◽

Differential Modulation ◽

Global And Local

Download Full-text

Neural ensemble coding in inferior temporal cortex

Journal of Neurophysiology ◽

10.1152/jn.1994.71.6.2325 ◽

1994 ◽

Vol 71 (6) ◽

pp. 2325-2337 ◽

Cited By ~ 91

Author(s):

P. M. Gochin ◽

M. Colombo ◽

G. A. Dorfman ◽

G. L. Gerstein ◽

C. G. Gross

Keyword(s):

Time Scale ◽

Visual Stimuli ◽

Temporal Cortex ◽

Population Characteristics ◽

Basis Set ◽

Inferior Temporal Cortex ◽

Stimulus Discrimination ◽

Linear Discriminant ◽

Neural Ensemble ◽

Discrimination Capacity

1. Isolated, single-neuron extracellular potentials were recorded sequentially in area TE of the inferior temporal cortex (IT) of two macaque monkeys (n = 58 and n = 41 neurons). Data were obtained while the animals were performing a paired-associate task. The task utilized five stimuli and eight stimulus pairings (4 correct and 4 incorrect). Data were evaluated as average spike rate during experimental epochs of 100 or 400 ms. Single-unit and population characteristics were measured using a form of linear discriminant analysis and information theoretic measures. To evaluate the significance of covariance on population code measures, additional data consisting of simultaneous recordings from < or = 8 isolated neurons (n = 37) were obtained from a third macaque monkey that was passively viewing visual stimuli. 2. On average, 43% of IT neurons were activated by any of the stimuli used (60% if those inhibited also are included). Yet the neurons were rather unique in the relative magnitude of their responses to each stimulus in the test set. These results suggest that information may be represented in IT by the pattern of activity across neurons and that the representation is not sparsely coded. It is further suggested that the representation scheme may have similarities to DNA or computer codes wherein a coding element is not a local parametric descriptor. This is a departure from the V1 representation, which appears to be both local and parametric. It is also different from theories of IT representation that suggest a constructive basis set or “alphabet”. From this view, determination of stimulus discrimination capacity in IT should be evaluated by measures of population activity patterns. 3. Evaluation of small groups of simultaneously recorded neurons obtained during a fixation task suggests that little information about visual stimuli is conveyed by covariance of activity in IT when a 100-ms time scale is used as in this study. This finding is consistent with a prior report, by Gochin et al., which used a 1-ms time scale and failed to find neural activity coherence or oscillations dependent on stimuli. 4. Population-stimulus-discrimination capacity measures were influenced by the number of neurons and to some extent the number and type of stimuli. 5. Information conveyed by individual neurons (mutual information) averaged 0.26 bits. The distribution of information values was unimodal and is therefore more consistent with a distributed than a local coding scheme.(ABSTRACT TRUNCATED AT 400 WORDS)

Download Full-text

Representation of visual stimuli in inferior temporal cortex

Philosophical Transactions of the Royal Society B Biological Sciences ◽

10.1098/rstb.1992.0001 ◽

1992 ◽

Vol 335 (1273) ◽

pp. 3-10 ◽

Cited By ~ 141

Keyword(s):

Visual Information ◽

Visual Stimuli ◽

Temporal Cortex ◽

Inferior Temporal Cortex ◽

Facial Image ◽

Stimulus Properties ◽

Visual Patterns ◽

Feature Detectors ◽

Processing And Storage ◽

And Storage

In primates, inferior temporal (IT) cortex is crucial for the processing and storage of visual information about form and colour. This article reviews the properties of IT neurons and considers how these properties may underlie the perceptual and mnemonic functions of IT cortex. The available evidence suggests that the processing of the facial image by IT cortex is similar to its processing of other visual patterns. Faces and other complex visual stimuli appear to be represented by the pattern of responses over a population of IT neurons rather than by the responses of specific ‘feature detectors’ or ‘grandmother’ cells. IT neurons with adult-like stimulus properties are present in monkeys as young as six weeks old.

Download Full-text