Object-scene conceptual regularities reveal fundamental differences between biological and artificial object vision

Human vision is still largely unexplained. Computer vision made impressive progress on this front, but it is unclear to what extent artificial neural networks approximate human brain strategies. Here, we confirm this gap by testing how biological and artificial systems encode object-scene contextual regularities in natural images. Both systems represent these regularities, but the underlying information processing is markedly different. In human vision, objects and backgrounds are represented separately, with rich domain-specific representations characterizing human visual cortex. Interaction between these components occurs downstream in frontoparietal areas. Conversely, neural networks represent image components in a single entangled representation revealing reduced object-segregation abilities and impoverished domain-specific object spaces. These results show the uniqueness of human vision that allows understanding that images are not just a collection of features and points to the need for developing neural network models with a similar richness of representational content.

Triggerfish uses chromaticity and lightness for object segregation

Humans group components of visual patterns according to their colour, and perceive colours separately from shape. This property of human visual perception is the basis behind the Ishihara test for colour deficiency, where an observer is asked to detect a pattern made up of dots of similar colour with variable lightness against a background of dots made from different colour(s) and lightness. To find out if fish use colour for object segregation in a similar manner to humans, we used stimuli inspired by the Ishihara test. Triggerfish ( Rhinecanthus aculeatus ) were trained to detect a cross constructed from similarly coloured dots against various backgrounds. Fish detected this cross even when it was camouflaged using either achromatic or chromatic noise, but fish relied more on chromatic cues for shape segregation. It remains unknown whether fish may switch to rely primarily on achromatic cues in scenarios where target objects have higher achromatic contrast and lower chromatic contrast. Fish were also able to generalize between stimuli of different colours, suggesting that colour and shape are processed by fish independently.

Depth perception in disparity-defined objects: finding the balance between averaging and segregation

Deciding what constitutes an object, and what background, is an essential task for the visual system. This presents a conundrum: averaging over the visual scene is required to obtain a precise signal for object segregation, but segregation is required to define the region over which averaging should take place. Depth, obtained via binocular disparity (the differences between two eyes’ views), could help with segregation by enabling identification of object and background via differences in depth. Here, we explore depth perception in disparity-defined objects. We show that a simple object segregation rule, followed by averaging over that segregated area, can account for depth estimation errors. To do this, we compared objects with smoothly varying depth edges to those with sharp depth edges, and found that perceived peak depth was reduced for the former. A computational model used a rule based on object shape to segregate and average over a central portion of the object, and was able to emulate the reduction in perceived depth. We also demonstrated that the segregated area is not predefined but is dependent on the object shape. We discuss how this segregation strategy could be employed by animals seeking to deter binocular predators. This article is part of the themed issue ‘Vision in our three-dimensional world’.

Neuronal Correlates of Multiple Top–Down Signals during Covert Tracking of Moving Objects in Macaque Prefrontal Cortex

Resistance to distraction is a key component of executive functions and is strongly linked to the prefrontal cortex. Recent evidence suggests that neural mechanisms exist for selective suppression of task-irrelevant information. However, neuronal signals related to selective suppression have not yet been identified, whereas nonselective surround suppression, which results from attentional enhancement for relevant stimuli, has been well documented. This study examined single neuron activities in the lateral PFC when monkeys covertly tracked one of randomly moving objects. Although many neurons responded to the target, we also found a group of neurons that exhibited a selective response to the distractor that was visually identical to the target. Because most neurons were insensitive to an additional distractor that explicitly differed in color from the target, the brain seemed to monitor the distractor only when necessary to maintain internal object segregation. Our results suggest that the lateral PFC might provide at least two top–down signals during covert object tracking: one for enhancement of visual processing for the target and the other for selective suppression of visual processing for the distractor. These signals might work together to discriminate objects, thereby regulating both the sensitivity and specificity of target choice during covert object tracking.