Previous visual experience affects aspects of later language processing

Incidental Visual Experience Affects Language Processing: An ERP Study

PsycEXTRA Dataset ◽

10.1037/e527312012-315 ◽

2008 ◽

Author(s):

Rolf A. Zwaan ◽

Leonora C. Coppens ◽

Liselotte Gootjes

Keyword(s):

Language Processing ◽

Visual Experience

Download Full-text

Modelling Form-Meaning Systematicity with Linguistic and Visual Features

Proceedings of the AAAI Conference on Artificial Intelligence ◽

10.1609/aaai.v34i05.6416 ◽

2020 ◽

Vol 34 (05) ◽

pp. 8870-8877

Author(s):

Arie Soeteman ◽

Dario Gutierrez ◽

Elia Bruni ◽

Ekaterina Shutova

Keyword(s):

Language Processing ◽

Visual Experience ◽

Word Meaning ◽

Data Driven ◽

Visual Features ◽

Multimodal Approach ◽

Word Form ◽

Word Meanings ◽

Textual Data ◽

Existing Data

Several studies in linguistics and natural language processing (NLP) pointed out systematic correspondences between word form and meaning in language. A prominent example of such systematicity is iconicity, which occurs when the form of a word is motivated by some perceptual (e.g. visual) aspect of its referent. However, the existing data-driven approaches to form-meaning systematicity modelled word meanings relying on information extracted from textual data alone. In this paper, we investigate to what extent our visual experience explains some of the form-meaning systematicity found in language. We construct word meaning representations from linguistic as well as visual data and analyze the structure and significance of form-meaning systematicity found in English using these models. Our findings corroborate the existence of form-meaning systematicity and show that this systematicity is concentrated in localized clusters. Furthermore, applying a multimodal approach allows us to identify new patterns of systematicity that have not been previously identified with the text-based models.

Download Full-text

Language processing is not a race against time

Behavioral and Brain Sciences ◽

10.1017/s0140525x15000692 ◽

2016 ◽

Vol 39 ◽

Cited By ~ 1

Author(s):

Giosuè Baggio ◽

Carmelo M. Vicario

Keyword(s):

Information Processing ◽

Human Brain ◽

Language Processing ◽

Cognitive Functions ◽

Memory Systems ◽

Language System ◽

Moderating Factors

AbstractWe agree with Christiansen & Chater (C&C) that language processing and acquisition are tightly constrained by the limits of sensory and memory systems. However, the human brain supports a range of cognitive functions that mitigate the effects of information processing bottlenecks. The language system is partly organised around these moderating factors, not just around restrictions on storage and computation.

Download Full-text

Cognitive Processing of Miscommunication in Interactive Listening: An Evaluation of Listener Indecision and Cognitive Effort

Journal of Speech Language and Hearing Research ◽

10.1044/2020_jslhr-20-00128 ◽

2021 ◽

pp. 1-17

Author(s):

Jennifer M. Roche ◽

Arkady Zgonnikov ◽

Laura M. Morett

Keyword(s):

Decision Making ◽

Cognitive Processing ◽

Language Processing ◽

Cognitive Effort ◽

Mouse Tracking ◽

Computer Mouse ◽

Decision Making Processes ◽

Computer Mouse Tracking ◽

Listening Task ◽

Processing Effort

Purpose The purpose of the current study was to evaluate the social and cognitive underpinnings of miscommunication during an interactive listening task. Method An eye and computer mouse–tracking visual-world paradigm was used to investigate how a listener's cognitive effort (local and global) and decision-making processes were affected by a speaker's use of ambiguity that led to a miscommunication. Results Experiments 1 and 2 found that an environmental cue that made a miscommunication more or less salient impacted listener language processing effort (eye-tracking). Experiment 2 also indicated that listeners may develop different processing heuristics dependent upon the speaker's use of ambiguity that led to a miscommunication, exerting a significant impact on cognition and decision making. We also found that perspective-taking effort and decision-making complexity metrics (computer mouse tracking) predict language processing effort, indicating that instances of miscommunication produced cognitive consequences of indecision, thinking, and cognitive pull. Conclusion Together, these results indicate that listeners behave both reciprocally and adaptively when miscommunications occur, but the way they respond is largely dependent upon the type of ambiguity and how often it is produced by the speaker.

Download Full-text