scholarly journals The Dynamics of Attention Shifts Among Concurrent Speech in a Naturalistic Multi-Speaker Virtual Environment

2019 ◽  
Author(s):  
Keren Shavit Cohen ◽  
Elana Zion Golumbic
Keyword(s):  
Virtual Reality ◽  
Eye Gaze ◽  
Real Life ◽  
Irrelevant Speech ◽  
Speech Comprehension ◽  
Gaze Shifts ◽  
Novel Approach ◽  
Entire Experiment ◽  
Task Irrelevant ◽  
Target Speaker

AbstractFocusing attention on one speaker on the background of other irrelevant speech can be a challenging feat. A longstanding question in attention research is whether and how frequently individuals shift their attention towards task-irrelevant speech, arguably leading to occasional detection of words in a so-called unattended message. However, this has been difficult to gauge empirically, particularly when participants attend to continuous natural speech, due to the lack of appropriate metrics for detecting shifts in internal attention. Here we introduce a new experimental platform for studying the dynamic deployment of attention among concurrent speakers, utilizing a unique combination of Virtual Reality and Eye-Tracking technology. We created a Virtual Café in which participants sit across from and attend to the narrative of a target speaker. We manipulate the number and location of distractor speakers, manifest as additional patrons throughout the Virtual Café. By monitoring participant’ eye-gaze dynamics, we studied the patterns of overt shifts of attention among the concurrent speakers as well as the consequences of these shifts on speech comprehension.Our results reveal important individual differences in the gaze-pattern displayed during selective attention to speech. While some participants stayed fixated on a target speaker throughout the entire experiment, approximately 30% of participants frequently shifted their gaze toward distractor speakers or other locations in the environment, regardless of the severity of audiovisual distraction. Critically, the tendency for frequent gaze-shifts negatively impacted comprehension of the target speaker. We also found that gaze-shifts occurred primarily during gaps in the acoustic input, suggesting they are prompted by momentary unmasking of the competing audio, in line with ‘glimpsing’ theories of processing speech in noise.These results open a new window into understanding the dynamics of attention as they wax and wane over time, and the different listening patterns employed for dealing with the influx of sensory input in multisensory environments. Moreover, the novel approach developed here for tracking the locus of momentary attention in a naturalistic virtual-reality environment holds high promise for extending the study of human behavior and cognition and bridging the gap between the laboratory and real-life.

scholarly journals Working-memory disruption by task-irrelevant talkers depends on degree of talker familiarity

2018 ◽  
Author(s):  
Jens Kreitewolf ◽  
Malte Wöstmann ◽  
Sarah Tune ◽  
Michael Plöchl ◽  
Jonas Obleser
Keyword(s):  
Working Memory ◽  
Family Members ◽  
Serial Order ◽  
Irrelevant Speech ◽  
Speech Comprehension ◽  
Memory Retention ◽  
Statistics Course ◽  
Memory Disruption ◽  
Task Irrelevant ◽  
Talker Familiarity

AbstractWhen listening, familiarity with an attended talker’s voice improves speech comprehension. Here, we instead investigated the effect of familiarity with a distracting talker. In an irrelevant-speech task, we assessed listeners’ working memory for the serial order of spoken digits when a task-irrelevant, distracting sentence was produced by either a familiar or an unfamiliar talker (with rare omissions of the task-irrelevant sentence). We tested two groups of listeners using the same experimental procedure. The first group were undergraduate psychology students (N=66) who had attended an introductory statistics course. Critically, each student had been taught by one of two course instructors, whose voices served as familiar and unfamiliar task-irrelevant talkers. The second group of listeners were family members and friends (N=20) who had known either one of the two talkers for more than ten years. Students, but not family members and friends, made more errors when the task-irrelevant talker was familiar versus unfamiliar. Interestingly, the effect of talker familiarity was not modulated by the presence of task-irrelevant speech: students experienced stronger working-memory disruption by a familiar talker irrespective of whether they heard a task-irrelevant sentence during memory retention or merely expected it. While previous work has shown that familiarity with an attended talker benefits speech comprehension, our findings indicate that familiarity with an ignored talker deteriorates working memory for target speech. The absence of this effect in family members and friends suggests that the degree of familiarity modulates memory disruption.

scholarly journals Eye Gaze And Perceptual Adaptation To Audiovisual Degraded Speech

2020 ◽  
Author(s):  
Briony Banks ◽  
Emma Gowen ◽  
Kevin Munro ◽  
patti adank
Keyword(s):  
Visual Cues ◽  
Eye Gaze ◽  
Real Life ◽  
Audio Signal ◽  
Current Evidence ◽  
Control Group ◽  
Speech Comprehension ◽  
Perceptual Adaptation ◽  
Degraded Speech ◽  
Over Time

Visual cues from a speaker’s face may improve perceptual adaptation to degraded speech over time, but current evidence is limited. We aimed to replicate results from previous studies and extend them to more demanding speech stimuli (sentences), to better represent real-life, challenging speech comprehension. In addition, we investigated whether particular eye gaze patterns towards the speaker’s mouth were related to adaptation, hypothesising that listeners who looked more at the speaker’s mouth would show greater adaptation. A group of listeners were presented with noise-vocoded sentences in audiovisual format while a control group were presented with the audio signal only, presented congruently with a still image of the speaker’s face. Results of previous adaptation studies were partially replicated: the audiovisual group had better recognition throughout and adapted slightly more rapidly, but both groups showed an equal amount of improvement overall (after exposure to 90 sentences). Longer fixations on the speaker’s mouth in the audiovisual group were related to better overall accuracy, although evidence for this relationship was relatively weak. An exploratory analysis further showed that the duration of fixations to the speaker’s mouth decreased over time. The results suggest that the benefits from visual cues to adaptation to unfamiliar speech vary more than previously thought. Longer fixations on a speaker’s mouth may play a role in successfully decoding these cues, but more evidence is needed to fully establish how patterns of eye gaze are related to audiovisual speech recognition.

scholarly journals Circular Economy and Virtual Reality in Advanced BIM-Based Prefabricated Construction

Energies ◽  
2021 ◽  
Vol 14 (13) ◽  
pp. 4065
Author(s):  
Timothy M. O’Grady ◽  
Nicholas Brajkovich ◽  
Roberto Minunno ◽  
Heap-Yih Chong ◽  
Gregory M. Morrison
Keyword(s):  
Virtual Reality ◽  
Construction Industry ◽  
Circular Economy ◽  
Game Design ◽  
Real Life ◽  
Case Study Methodology ◽  
Novel Approach ◽  
Living Lab ◽  
Building Information

This paper presents a new virtual reality (VR)-based approach to advanced learnings and experiences of the circular economy (CE) in the construction industry. The approach involves incorporating game design and a building information modelling (BIM) digital twin of a purposed CE prototype building. Our novel approach introduces VR environments designed to provide a visual representation of materials and components that can be reintroduced into the supply chain at the end of life and their removal procedures and material provenance. A case study methodology was applied to a purposely designed CE building, namely the Legacy Living Lab (L3). To reflect the real-life building, L3’s BIM model was combined with Unify game software to advance the literature in three key areas. First, the research investigates VR tools that will allow building designers to view and implement their strategies to advance CE design. Second, this research proposes an advanced VR tool to visualise the bill of quantities (BoQ) and material stock embedded in the studied building, further understanding concepts such as buildings as material banks. Finally, the proposed VR environment defines CE techniques implemented within the case study to be disseminated across the vast construction industry. This VR research identifies three key pillars in reducing the waste generated by the construction industry: education, documentation and visualisation. Furthermore, this paper provides a visual link between the BIM, BoQ and resiliency of the selected materials.

scholarly journals New fractional approaches for n-polynomial P-convexity with applications in special function theory

2020 ◽  
Vol 2020 (1) ◽  
Author(s):  
Shu-Bo Chen ◽  
Saima Rashid ◽  
Muhammad Aslam Noor ◽  
Zakia Hammouch ◽  
Yu-Ming Chu
Keyword(s):  
Fractional Calculus ◽  
Convex Functions ◽  
Special Functions ◽  
Real Life ◽  
Fractional Integrals ◽  
Digamma Function ◽  
Novel Approach ◽  
Inequality Theory ◽  
New Strategy ◽  
Significant Mechanism

Abstract Inequality theory provides a significant mechanism for managing symmetrical aspects in real-life circumstances. The renowned distinguishing feature of integral inequalities and fractional calculus has a solid possibility to regulate continuous issues with high proficiency. This manuscript contributes to a captivating association of fractional calculus, special functions and convex functions. The authors develop a novel approach for investigating a new class of convex functions which is known as an n-polynomial $\mathcal{P}$ P -convex function. Meanwhile, considering two identities via generalized fractional integrals, provide several generalizations of the Hermite–Hadamard and Ostrowski type inequalities by employing the better approaches of Hölder and power-mean inequalities. By this new strategy, using the concept of n-polynomial $\mathcal{P}$ P -convexity we can evaluate several other classes of n-polynomial harmonically convex, n-polynomial convex, classical harmonically convex and classical convex functions as particular cases. In order to investigate the efficiency and supremacy of the suggested scheme regarding the fractional calculus, special functions and n-polynomial $\mathcal{P}$ P -convexity, we present two applications for the modified Bessel function and $\mathfrak{q}$ q -digamma function. Finally, these outcomes can evaluate the possible symmetric roles of the criterion that express the real phenomena of the problem.

scholarly journals Assessment of Wheelchair Propulsion Performance in an Immersive Virtual Reality Simulator

2021 ◽  
Vol 18 (15) ◽  
pp. 8016
Author(s):  
Yu-Sheng Yang ◽  
Alicia M. Koontz ◽  
Yu-Hsuan Hsiao ◽  
Cheng-Tang Pan ◽  
Jyh-Jong Chang
Keyword(s):  
Virtual Reality ◽  
Spinal Cord Injuries ◽  
Qualitative Interviews ◽  
Real Life ◽  
Structured Interview ◽  
Wheelchair Propulsion ◽  
Subjective Data ◽  
Wheelchair Users ◽  
Head Mounted Display ◽  
Propulsion Performance

Maneuvering a wheelchair is an important necessity for the everyday life and social activities of people with a range of physical disabilities. However, in real life, wheelchair users face several common challenges: articulate steering, spatial relationships, and negotiating obstacles. Therefore, our research group has developed a head-mounted display (HMD)-based intuitive virtual reality (VR) stimulator for wheelchair propulsion. The aim of this study was to investigate the feasibility and efficacy of this VR stimulator for wheelchair propulsion performance. Twenty manual wheelchair users (16 men and 4 women) with spinal cord injuries ranging from T8 to L2 participated in this study. The differences in wheelchair propulsion kinematics between immersive and non-immersive VR environments were assessed using a 3D motion analysis system. Subjective data of the HMD-based intuitive VR stimulator were collected with a Presence Questionnaire and individual semi-structured interview at the end of the trial. Results indicated that propulsion performance was very similar in terms of start angle (p = 0.34), end angle (p = 0.46), stroke angle (p = 0.76), and shoulder movement (p = 0.66) between immersive and non-immersive VR environments. In the VR episode featuring an uphill journey, an increase in propulsion speed (p < 0.01) and cadence (p < 0.01) were found, as well as a greater trunk forward inclination (p = 0.01). Qualitative interviews showed that this VR simulator made an attractive, novel impression and therefore demonstrated the potential as a tool for stimulating training motivation. This HMD-based intuitive VR stimulator can be an effective resource to enhance wheelchair maneuverability experiences.

New Multi-View Classification Method with Uncertain Data

2021 ◽  
Vol 16 (1) ◽  
pp. 1-23
Author(s):  
Bo Liu ◽  
Haowen Zhong ◽  
Yanshan Xiao
Keyword(s):  
Learning Strategy ◽  
State Of The Art ◽  
Uncertain Data ◽  
Real Life ◽  
Support Vector ◽  
Classification Methods ◽  
Complementary Information ◽  
Novel Approach ◽  
Svm Model ◽  
Iterative Framework

Multi-view classification aims at designing a multi-view learning strategy to train a classifier from multi-view data, which are easily collected in practice. Most of the existing works focus on multi-view classification by assuming the multi-view data are collected with precise information. However, we always collect the uncertain multi-view data due to the collection process is corrupted with noise in real-life application. In this case, this article proposes a novel approach, called uncertain multi-view learning with support vector machine (UMV-SVM) to cope with the problem of multi-view learning with uncertain data. The method first enforces the agreement among all the views to seek complementary information of multi-view data and takes the uncertainty of the multi-view data into consideration by modeling reachability area of the noise. Then it proposes an iterative framework to solve the proposed UMV-SVM model such that we can obtain the multi-view classifier for prediction. Extensive experiments on real-life datasets have shown that the proposed UMV-SVM can achieve a better performance for uncertain multi-view classification in comparison to the state-of-the-art multi-view classification methods.

scholarly journals Presence Effects in Virtual Reality Based on User Characteristics: Attention, Enjoyment, and Memory

Electronics ◽  
2021 ◽  
Vol 10 (9) ◽  
pp. 1051
Author(s):  
Si Jung Kim ◽  
Teemu H. Laine ◽  
Hae Jung Suk
Keyword(s):  
Virtual Reality ◽  
Emotional State ◽  
Eye Gaze ◽  
Emotional Valence ◽  
Media Content ◽  
Eye Tracker ◽  
User Characteristics ◽  
Head Mounted Display ◽  
Age Range ◽  
Different Levels

Presence refers to the emotional state of users where their motivation for thinking and acting arises based on the perception of the entities in a virtual world. The immersion level of users can vary when they interact with different media content, which may result in different levels of presence especially in a virtual reality (VR) environment. This study investigates how user characteristics, such as gender, immersion level, and emotional valence on VR, are related to the three elements of presence effects (attention, enjoyment, and memory). A VR story was created and used as an immersive stimulus in an experiment, which was presented through a head-mounted display (HMD) equipped with an eye tracker that collected the participants’ eye gaze data during the experiment. A total of 53 university students (26 females, 27 males), with an age range from 20 to 29 years old (mean 23.8), participated in the experiment. A set of pre- and post-questionnaires were used as a subjective measure to support the evidence of relationships among the presence effects and user characteristics. The results showed that user characteristics, such as gender, immersion level, and emotional valence, affected their level of presence, however, there is no evidence that attention is associated with enjoyment or memory.

Sign in / Sign up

Export Citation Format

Share Document