Capturing the response dynamics of attention capture with mouse tracking

Human behavior is often remarkably flexible, showing the ability to quickly adapt to the statistical peculiarities of a particular local context. When it comes to language, previous work has shown that listeners' anticipatory interpretations of intonational cues are adapted dynamically when cues are observed to be stochastically unreliable. This paper reports novel empirical data from manual response dynamics (mouse-tracking) on how listeners adapt their predictive interpretation when some intonational cues are occasionally unreliable while others are consistently reliable. A model of rational belief dynamics predicts that listeners adapt differently to different unreliable intonational cues, as a function of their initial evidential strength. These predictions are borne out by our data.

Download Full-text

Testing the effectiveness of mouse tracking in speech perception

10.31237/osf.io/zkaqy ◽

2019 ◽

Author(s):

Mathias Stoeber

Keyword(s):

Speech Perception ◽

Categorical Perception ◽

Motor Behavior ◽

Mouse Tracking ◽

Response Dynamics ◽

Manual Response ◽

Phonological Representations ◽

Dimensional Measurements ◽

On Line ◽

Category Variability

Listeners routinely perceive phonetic speech signals which are made up of acoustic detail belonging to multiple continuous physical dimensions (e.g. intensity, frequency, duration), and then discretely map them onto phonological units like phonemes with ease. Traditional accounts of speech perception suggest that listeners achieve this by discarding all non-distinctive (within-category) variability in the signal in favor of discrete phonological representations, resulting in a phenomenon known as categorical perception. However, more recent findings show that listeners do exhibit sensitivity to intra-categorical phonetic detail, for example by investigating on-line measurements from eye-tracking. It is yet unknown whether mouse tracking, a nascent experimental method capable of producing continuous multi-dimensional measurements from motor behavior, can similarly contribute meaningful evidence to research into categorical perception of speech sounds. The present exploratory studies indicate that the effectiveness of the mouse tracking paradigm may be severely limited for such purposes. Here, mouse tracking was unable to replicate previous findings on listener sensitivity to sub-phonemic variability, although a decidedly non-antagonistic replication attempt was made with regards to the design specifics of the paradigm. These findings undermine assumptions about the mapping between cognitive processes and manual response dynamics, questioning the utility of mouse tracking for speech perception research.

Download Full-text

Cognitive Processing of Miscommunication in Interactive Listening: An Evaluation of Listener Indecision and Cognitive Effort

Journal of Speech Language and Hearing Research ◽

10.1044/2020_jslhr-20-00128 ◽

2021 ◽

pp. 1-17

Author(s):

Jennifer M. Roche ◽

Arkady Zgonnikov ◽

Laura M. Morett

Keyword(s):

Decision Making ◽

Cognitive Processing ◽

Language Processing ◽

Cognitive Effort ◽

Mouse Tracking ◽

Computer Mouse ◽

Decision Making Processes ◽

Computer Mouse Tracking ◽

Listening Task ◽

Processing Effort

Purpose The purpose of the current study was to evaluate the social and cognitive underpinnings of miscommunication during an interactive listening task. Method An eye and computer mouse–tracking visual-world paradigm was used to investigate how a listener's cognitive effort (local and global) and decision-making processes were affected by a speaker's use of ambiguity that led to a miscommunication. Results Experiments 1 and 2 found that an environmental cue that made a miscommunication more or less salient impacted listener language processing effort (eye-tracking). Experiment 2 also indicated that listeners may develop different processing heuristics dependent upon the speaker's use of ambiguity that led to a miscommunication, exerting a significant impact on cognition and decision making. We also found that perspective-taking effort and decision-making complexity metrics (computer mouse tracking) predict language processing effort, indicating that instances of miscommunication produced cognitive consequences of indecision, thinking, and cognitive pull. Conclusion Together, these results indicate that listeners behave both reciprocally and adaptively when miscommunications occur, but the way they respond is largely dependent upon the type of ambiguity and how often it is produced by the speaker.

Download Full-text