(Eye-)Tracking the time-course of the interaction between linguistic and visual processes: the effect of verb-conceptual restrictions

AbstractThe ability to predict what is going to happen in the near future is integral for daily functioning. Previous research suggests that predictability varies over time, with increases in prediction error at those moments that people perceive as boundaries between meaningful events. These moments also tend to be points of rapid change in the environment. Eye tracking provides a method for continuous measurement of prediction as participants watch a movie of an actor performing a series of actions. In two studies, we used eye tracking to study the time course of prediction around event boundaries. In both studies, viewers looked at objects that were about to be touched by the actor shortly before the objects were contacted, demonstrating predictive looking. However, this behavior was modulated by event boundaries: looks to to-be-contacted objects near event boundaries were less likely to be early and more likely to be late, compared to looks to objects contacted within events. This result is consistent with theories proposing that event segmentation results from transient increases in prediction error.Significance StatementThe ability to predict what will happen in the near future is integral for adaptive functioning, and although there has been extensive research on predictive processing, the dynamics of prediction at the second by second level during the perception of naturalistic activity has never been explored. The current studies therefore describe results from a novel task, the Predictive Looking at Action Task (PLAT) that can be used to investigate the dynamics of predictive processing. Demonstrating the utility of this task to investigate predictive processing, this task was applied to study the predictions made by Event Segmentation Theory, which suggests that people experience event boundaries at times of change and unpredictability in the environment. The results of these studies are of interest to communities investigating the dynamic comprehension and segmentation of naturalistic events and to communities studying visual perception of naturalistic activity.

Download Full-text

The Interaction of Morphological Gender With Stereotypical Information: An Eye Tracking Study on Gender Inferences

International Journal of Linguistics ◽

10.5296/ijl.v11i4.15169 ◽

2019 ◽

Vol 11 (4) ◽

pp. 111

Author(s):

Daniela Ronca ◽

Vincenzo Moscati

Keyword(s):

Eye Tracking ◽

Gender Stereotypes ◽

Mental Model ◽

Time Course ◽

Sentence Comprehension ◽

Visual World ◽

Male And Female ◽

Gender Biases ◽

On Line ◽

Competing Models

In this paper we observe the time-course and the activation of gender stereotypes comparing the predictions of two competing models: the Minimalist (McKoon et al., 1992) and the Mental Model Hypothesis (Garnham 2001). The on-line processing of sentences containing male-biased stereotypes is experimentally investigated in Italian on epicenes nouns (i.e. nouns that do not morphologically disambiguate between male and female referents) adopting a procedure based on the Visual World paradigm.Eye-movements during sentence comprehension show that stereotypes become immediately active as soon as male-biased role nouns are encountered, as predicted by the Mental Model Hypothesis. Our results also show that when disambiguating cues based on morphological agreement are provided, the activation of stereotypes is blocked. This indicates that morphological gender is quickly processed and that it can suppress stereotypical gender biases.

Download Full-text

The Roles of Consonant, Rime, and Tone in Mandarin Spoken Word Recognition: An Eye-Tracking Study

Frontiers in Psychology ◽

10.3389/fpsyg.2021.740444 ◽

2022 ◽

Vol 12 ◽

Author(s):

Ting Zou ◽

Yutong Liu ◽

Huiting Zhong

Keyword(s):

Word Recognition ◽

Eye Tracking ◽

Time Course ◽

Spoken Word Recognition ◽

Continuous Mapping ◽

Phonological Similarity ◽

Spoken Word ◽

Relative Role ◽

Growth Curve Analysis ◽

Mapping Models

This study investigated the relative role of sub-syllabic components (initial consonant, rime, and tone) in spoken word recognition of Mandarin Chinese using an eye-tracking experiment with a visual world paradigm. Native Mandarin speakers (all born and grew up in Beijing) were presented with four pictures and an auditory stimulus. They were required to click the picture according to the sound stimulus they heard, and their eye movements were tracked during this process. For a target word (e.g., tang2 “candy”), nine conditions of competitors were constructed in terms of the amount of their phonological overlap with the target: consonant competitor (e.g., ti1 “ladder”), rime competitor (e.g., lang4 “wave”), tone competitor (e.g., niu2 “cow”), consonant plus rime competitor (e.g., tang1”soup”), consonant plus tone competitor (e.g., tou2 “head”), rime plus tone competitor (e.g., yang2 “sheep”), cohort competitor (e.g., ta3 “tower”), cohort plus tone competitor (e.g., tao2 “peach”), and baseline competitor (e.g., xue3 “snow”). A growth curve analysis was conducted with the fixation to competitors, targets, and distractors, and the results showed that (1) competitors with consonant or rime overlap can be adequately activated, while tone overlap plays a weaker role since additional tonal information can strengthen the competitive effect only when it was added to a candidate that already bears much phonological similarity with the target. (2) Mandarin words are processed in an incremental way in the time course of word recognition since different partially overlapping competitors could be activated immediately; (3) like the pattern found in English, both cohort and rime competitors were activated to compete for lexical activation, but these two competitors were not temporally distinctive and mainly differed in the size of their competitive effects. Generally, the gradation of activation based on the phonological similarity between target and candidates found in this study was in line with the continuous mapping models and may reflect a strategy of native speakers shaped by the informative characteristics of the interaction among different sub-syllabic components.

Download Full-text

Do Musicians and Non-musicians Differ in Speech-on-Speech Processing?

Frontiers in Psychology ◽

10.3389/fpsyg.2021.623787 ◽

2021 ◽

Vol 12 ◽

Author(s):

Elif Canseza Kaplan ◽

Anita E. Wagner ◽

Paolo Toffanin ◽

Deniz Başkent

Keyword(s):

Speech Perception ◽

Eye Tracking ◽

Language Processing ◽

Speech Processing ◽

Time Course ◽

Spoken Language ◽

Recall Task ◽

Spoken Language Processing ◽

Sentence Recall ◽

Gaze Fixations

Earlier studies have shown that musically trained individuals may have a benefit in adverse listening situations when compared to non-musicians, especially in speech-on-speech perception. However, the literature provides mostly conflicting results. In the current study, by employing different measures of spoken language processing, we aimed to test whether we could capture potential differences between musicians and non-musicians in speech-on-speech processing. We used an offline measure of speech perception (sentence recall task), which reveals a post-task response, and online measures of real time spoken language processing: gaze-tracking and pupillometry. We used stimuli of comparable complexity across both paradigms and tested the same groups of participants. In the sentence recall task, musicians recalled more words correctly than non-musicians. In the eye-tracking experiment, both groups showed reduced fixations to the target and competitor words’ images as the level of speech maskers increased. The time course of gaze fixations to the competitor did not differ between groups in the speech-in-quiet condition, while the time course dynamics did differ between groups as the two-talker masker was added to the target signal. As the level of two-talker masker increased, musicians showed reduced lexical competition as indicated by the gaze fixations to the competitor. The pupil dilation data showed differences mainly in one target-to-masker ratio. This does not allow to draw conclusions regarding potential differences in the use of cognitive resources between groups. Overall, the eye-tracking measure enabled us to observe that musicians may be using a different strategy than non-musicians to attain spoken word recognition as the noise level increased. However, further investigation with more fine-grained alignment between the processes captured by online and offline measures is necessary to establish whether musicians differ due to better cognitive control or sound processing.

Download Full-text

Eye tracking reveals expertise-related differences in the time-course of medical image inspection and diagnosis

Journal of Medical Imaging ◽

10.1117/1.jmi.7.5.051203 ◽

2020 ◽

Vol 7 (05) ◽

Author(s):

Tad T. Brunyé ◽

Trafton Drew ◽

Kathleen F. Kerr ◽

Hannah Shucard ◽

Donald L. Weaver ◽

...

Keyword(s):

Eye Tracking ◽

Medical Image ◽

Time Course ◽

Image Inspection

Download Full-text

Craving is everything: An eye-tracking exploration of attentional bias in binge drinking

Journal of Psychopharmacology ◽

10.1177/0269881120913131 ◽

2020 ◽

Vol 34 (6) ◽

pp. 636-647 ◽

Cited By ~ 2

Author(s):

Zoé Bollen ◽

Nicolas Masson ◽

Samuel Salvaggio ◽

Fabien D’Hondt ◽

Pierre Maurage

Keyword(s):

Alcohol Consumption ◽

Eye Tracking ◽

Binge Drinking ◽

Attentional Bias ◽

Time Course ◽

Soft Drink ◽

Consumption Pattern ◽

Area Of Interest ◽

Binge Drinkers ◽

High Calorie

Background: Attentional bias towards alcohol-related stimuli is a core characteristic of severe alcohol use disorders (AUD), directly linked to clinical variables (e.g. alcohol consumption, relapse). Nevertheless, the extent of this bias in subclinical populations remains poorly documented. This is particularly true for binge drinking, an alcohol consumption pattern highly prevalent in youth, characterised by an alternation between excessive intakes and withdrawal periods. Aims: We used eye-tracking to: (a) measure attentional bias in binge drinking, (b) determine its time course by dissociating early/late processing stages, (c) clarify its specificity for alcohol-related stimuli compared to other appetitive stimulations and (d) explore its modulation by current craving intensity. Methods: Binge drinkers ( n=42) and matched controls ( n=43) performed a visual probe task, requiring visual targets preceded by pairs of pictures to be processed, with three conditions (i.e. alcohol vs. soft drink, alcohol vs. high-calorie food, high-calorie food vs. low-calorie food). Results: No group difference was observed for early processing (i.e. first area of interest visited). Dwell times highlighted a bias towards soft drinks and healthy food among controls, without any global bias towards alcohol in binge drinkers. Centrally, a comparison of binge drinkers with low versus high current craving intensity indicated that binge drinking was associated with a bias towards alcohol and high-calorie food only in the presence of a high craving towards these stimuli. Conclusion: Attentional bias towards alcohol reported in severe AUD is only found in binge drinkers in the presence of high craving and is generalised to other appetitive cues.

Download Full-text