An image computable model of visual shape similarity

Yaniv Morgenstern; Filipp Schmidt; Frieder Hartmann; Henning Tiedemann; Eugen Prokott; Guido Maiello; Roland Fleming

doi:10.1167/19.10.37c

An image-computable model of human visual shape similarity

PLoS Computational Biology ◽

10.1371/journal.pcbi.1008981 ◽

2021 ◽

Vol 17 (6) ◽

pp. e1008981

Author(s):

Yaniv Morgenstern ◽

Frieder Hartmann ◽

Filipp Schmidt ◽

Henning Tiedemann ◽

Eugen Prokott ◽

...

Keyword(s):

Shape Similarity ◽

Computable Model ◽

Object Representations ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Wide Range ◽

Invaluable Tool ◽

Different Shapes ◽

Similarity Judgments ◽

Visual Shape

Shape is a defining feature of objects, and human observers can effortlessly compare shapes to determine how similar they are. Yet, to date, no image-computable model can predict how visually similar or different shapes appear. Such a model would be an invaluable tool for neuroscientists and could provide insights into computations underlying human shape perception. To address this need, we developed a model (‘ShapeComp’), based on over 100 shape features (e.g., area, compactness, Fourier descriptors). When trained to capture the variance in a database of >25,000 animal silhouettes, ShapeComp accurately predicts human shape similarity judgments between pairs of shapes without fitting any parameters to human data. To test the model, we created carefully selected arrays of complex novel shapes using a Generative Adversarial Network trained on the animal silhouettes, which we presented to observers in a wide range of tasks. Our findings show that incorporating multiple ShapeComp dimensions facilitates the prediction of human shape similarity across a small number of shapes, and also captures much of the variance in the multiple arrangements of many shapes. ShapeComp outperforms both conventional pixel-based metrics and state-of-the-art convolutional neural networks, and can also be used to generate perceptually uniform stimulus sets, making it a powerful tool for investigating shape and object representations in the human brain.

Download Full-text

An image-computable model of human visual shape similarity

10.1101/2020.01.10.901876 ◽

2020 ◽

Cited By ~ 2

Author(s):

Yaniv Morgenstern ◽

Frieder Hartmann ◽

Filipp Schmidt ◽

Henning Tiedemann ◽

Eugen Prokott ◽

...

Keyword(s):

Shape Similarity ◽

Computable Model ◽

Object Representations ◽

Shape Features ◽

Generative Adversarial Network ◽

Adversarial Network ◽

Wide Range ◽

Different Shapes ◽

Similarity Judgments ◽

Visual Shape

AbstractShape is a defining feature of objects. Yet, no image-computable model accurately predicts how similar or different shapes appear to human observers. To address this, we developed a model (‘ShapeComp’), based on over 100 shape features (e.g., area, compactness, Fourier descriptors). When trained to capture the variance in a database of >25,000 animal silhouettes, ShapeComp predicts human shape similarity judgments almost perfectly (r2>0.99) without fitting any parameters to human data. To test the model, we created carefully selected arrays of complex novel shapes using a Generative Adversarial Network trained on the animal silhouettes, which we presented to observers in a wide range of tasks. Our findings show that human shape perception is inherently multidimensional and optimized for comparing natural shapes. ShapeComp outperforms conventional metrics, and can also be used to generate perceptually uniform stimulus sets, making it a powerful tool for investigating shape and object representations in the human brain.

Download Full-text

Discrimination and Generalization of Complex Visual Shape Variations in Pigeons

Perceptual and Motor Skills ◽

10.2466/pms.1972.35.3.915 ◽

1972 ◽

Vol 35 (3) ◽

pp. 915-927 ◽

Cited By ~ 3

Author(s):

Douglas P. Ferraro ◽

Michael G. Grisham

Keyword(s):

Selective Attention ◽

Discrimination Learning ◽

Stimulus Control ◽

Complex Shape ◽

Shape Similarity ◽

Angular Orientation ◽

Functional Representation ◽

Key Pecking ◽

Generalization Gradients ◽

Visual Shape

Three experiments investigated stimulus control of key pecking in pigeons by varying the distance of vertices movement for a six-point complex visual shape. Ease of discrimination learning was monotonically related to the distance of vertices movement when the directions of vertices movement were held constant. As suggested by selective attention theory, steep generalization gradients were obtained following intradimensional differential training but not following nondifferential training or interdimensional differential training. These results indicate that, unlike the dimension of angular orientation or tilt, distance of vertices movement provides a consistent functional representation of complex shape similarity.

Download Full-text