Evaluation of human error of response to auditory and visual signals in the virtual reality

Toward Predicting Motion Sickness Using Virtual Reality and a Moving Platform Assessing Brain, Muscles, and Heart Signals

Frontiers in Bioengineering and Biotechnology ◽

10.3389/fbioe.2021.635661 ◽

2021 ◽

Vol 9 ◽

Author(s):

Marco Recenti ◽

Carlo Ricciardi ◽

Romain Aubonnet ◽

Ilaria Picone ◽

Deborah Jacob ◽

...

Keyword(s):

Virtual Reality ◽

Motion Sickness ◽

Muscle Activation ◽

Brain Activity ◽

Conflict Theory ◽

Visual Signals ◽

Physiological Factors ◽

Biomedical Sensors ◽

Dry Electrode ◽

Importance Analysis

Motion sickness (MS) and postural control (PC) conditions are common complaints among those who passively travel. Many theories explaining a probable cause for MS have been proposed but the most prominent is the sensory conflict theory, stating that a mismatch between vestibular and visual signals causes MS. Few measurements have been made to understand and quantify the interplay between muscle activation, brain activity, and heart behavior during this condition. We introduce here a novel multimetric system called BioVRSea based on virtual reality (VR), a mechanical platform and several biomedical sensors to study the physiology associated with MS and seasickness. This study reports the results from 28 individuals: the subjects stand on the platform wearing VR goggles, a 64-channel EEG dry-electrode cap, two EMG sensors on the gastrocnemius muscles, and a sensor on the chest that captures the heart rate (HR). The virtual environment shows a boat surrounded by waves whose frequency and amplitude are synchronized with the platform movement. Three measurement protocols are performed by each subject, after each of which they answer the Motion Sickness Susceptibility Questionnaire. Nineteen parameters are extracted from the biomedical sensors (5 from EEG, 12 from EMG and, 2 from HR) and 13 from the questionnaire. Eight binary indexes are computed to quantify the symptoms combining all of them in the Motion Sickness Index (IMS). These parameters create the MS database composed of 83 measurements. All indexes undergo univariate statistical analysis, with EMG parameters being most significant, in contrast to EEG parameters. Machine learning (ML) gives good results in the classification of the binary indexes, finding random forest to be the best algorithm (accuracy of 74.7 for IMS). The feature importance analysis showed that muscle parameters are the most relevant, and for EEG analysis, beta wave results were the most important. The present work serves as the first step in identifying the key physiological factors that differentiate those who suffer from MS from those who do not using the novel BioVRSea system. Coupled with ML, BioVRSea is of value in the evaluation of PC disruptions, which are among the most disturbing and costly health conditions affecting humans.

Download Full-text

Visual delay affects force scaling and weight perception during object lifting in virtual reality

Journal of Neurophysiology ◽

10.1152/jn.00396.2018 ◽

2019 ◽

Vol 121 (4) ◽

pp. 1398-1409 ◽

Cited By ~ 10

Author(s):

Vonne van Polanen ◽

Robert Tibold ◽

Atsuo Nuruki ◽

Marco Davare

Keyword(s):

Virtual Reality ◽

Multisensory Integration ◽

Force Feedback ◽

Haptic Feedback ◽

Visual Signals ◽

Visual Events ◽

Object Lifting ◽

Relative Weighting ◽

The Brain ◽

Visual Delay

Lifting an object requires precise scaling of fingertip forces based on a prediction of object weight. At object contact, a series of tactile and visual events arise that need to be rapidly processed online to fine-tune the planned motor commands for lifting the object. The brain mechanisms underlying multisensory integration serially at transient sensorimotor events, a general feature of actions requiring hand-object interactions, are not yet understood. In this study we tested the relative weighting between haptic and visual signals when they are integrated online into the motor command. We used a new virtual reality setup to desynchronize visual feedback from haptics, which allowed us to probe the relative contribution of haptics and vision in driving participants’ movements when they grasped virtual objects simulated by two force-feedback robots. We found that visual delay changed the profile of fingertip force generation and led participants to perceive objects as heavier than when lifts were performed without visual delay. We further modeled the effect of vision on motor output by manipulating the extent to which delayed visual events could bias the force profile, which allowed us to determine the specific weighting the brain assigns to haptics and vision. Our results show for the first time how visuo-haptic integration is processed at discrete sensorimotor events for controlling object-lifting dynamics and further highlight the organization of multisensory signals online for controlling action and perception. NEW & NOTEWORTHY Dexterous hand movements require rapid integration of information from different senses, in particular touch and vision, at different key time points as movement unfolds. The relative weighting between vision and haptics for object manipulation is unknown. We used object lifting in virtual reality to desynchronize visual and haptic feedback and find out their relative weightings. Our findings shed light on how rapid multisensory integration is processed over a series of discrete sensorimotor control points.

Download Full-text

Cyberpsychological Approach to the Analysis of Multisensory Integration

Counseling Psychology and Psychotherapy ◽

10.17759/cpp.2019270302 ◽

2019 ◽

Vol 27 (3) ◽

pp. 9-21

Author(s):

A.E. Voiskounsky

Keyword(s):

Risk Factors ◽

Virtual Reality ◽

Multisensory Integration ◽

Specific Area ◽

Visual Signals ◽

Sensory Conflict ◽

Body Experience ◽

Out Of Body Experience ◽

Transport Vehicle ◽

The Brain

The paper relates to the branch of cyberpsychology associated with risk factors during immersion in a virtual environment. Specialists in the development and operation of virtual reality systems know that immersion into this environment may be accompanied by symptoms similar to the “motion sickness” of transport vehicle passengers (ships, aircraft, cars). In the paper, these conditions are referred to as a cybersickness (or, cyberdisease). The three leading theories, proposed as an explanation of the causes of cybersickness, are discussed: the theory of sensory conflict, the theory of postural instability (the inability to maintain equilibrium), and the evolutionary (aka toxin) theory. A frequent occurrence of symptoms of cybersickness is a conflict between visual signals and signals from the vestibular system. It is shown that such conflicts can be stimulated in the framework of a specially organized experiment (e.g., the illusion of out-of-body experience) using virtual reality systems. When competing signals (visual, auditory, kinesthetic, tactile, etc.) reach the brain, the data gained with the use of virtual reality systems give a chance to hypothetically determine the localization of the specific area in the brain that ensures the integration of multisensory stimuli.

Download Full-text

Visual delay affects force scaling and weight perception when lifting objects in virtual reality: Supplemental model code

10.1101/504563 ◽

2018 ◽

Author(s):

Vonne van Polanen ◽

Robert Tibold ◽

Atsuo Nuruki ◽

Marco Davare

Keyword(s):

Virtual Reality ◽

Force Feedback ◽

Weight Perception ◽

Motor Command ◽

Visual Signals ◽

Model Code ◽

Relative Contribution ◽

Visual Events ◽

The Brain ◽

Visual Delay

Lifting an object requires precise scaling of fingertip forces based on a prediction of object weight. At object contact, a series of tactile and visual events arise that need to be rapidly processed online to fine-tune the planned motor commands for lifting the object. The brain mechanisms underlying multisensory integration serially at transient sensorimotor events, a general feature of actions requiring hand-object interactions, are not yet understood. Here we tested the relative weighting between haptic and visual signals when they are integrated online into the motor command. We used a new virtual reality setup to desynchronize visual feedback from haptics, which allowed us to probe the relative contribution of haptics and vision in driving participants' movements when they grasped virtual objects simulated by two force-feedback robots. We found that visual delay changed the profile of fingertip force generation and led participants to perceive objects as heavier than when lifts were performed without visual delay. We further modeled the effect of vision on motor output by manipulating the extent to which delayed visual events could bias the force profile, which allowed us to determine the specific weighting the brain assigns to haptics and vision. Our results show for the first time how visuo-haptic integration is processed at discrete sensorimotor events for controlling object lifting dynamics and further highlight the organization of multisensory signals online for controlling action and perception.

Download Full-text

Detection of neuronal response in a new mechatronics equipment with virtual reality images

Electroencephalography and Clinical Neurophysiology ◽

10.1016/s0013-4694(97)89078-9 ◽

1997 ◽

Vol 103 (1) ◽

pp. 223

Author(s):

A Rovetta

Keyword(s):

Virtual Reality ◽

Neuronal Response

Download Full-text

Top-Down Processing and Visual Reorientation Illusions in a Virtual Reality Environment

Swiss Journal of Psychology ◽

10.1024/1421-0185.63.3.143 ◽

2004 ◽

Vol 63 (3) ◽

pp. 143-149 ◽

Cited By ~ 3

Author(s):

Fred W. Mast ◽

Charles M. Oman

Keyword(s):

Virtual Reality ◽

Visual Processing ◽

Line Length ◽

Perceptual Cues ◽

Virtual Reality Environment ◽

Top Down ◽

Test Conditions ◽

The Subject ◽

Processing Mechanisms

The role of top-down processing on the horizontal-vertical line length illusion was examined by means of an ambiguous room with dual visual verticals. In one of the test conditions, the subjects were cued to one of the two verticals and were instructed to cognitively reassign the apparent vertical to the cued orientation. When they have mentally adjusted their perception, two lines in a plus sign configuration appeared and the subjects had to evaluate which line was longer. The results showed that the line length appeared longer when it was aligned with the direction of the vertical currently perceived by the subject. This study provides a demonstration that top-down processing influences lower level visual processing mechanisms. In another test condition, the subjects had all perceptual cues available and the influence was even stronger.

Download Full-text

Laparoscopic Cholecystectomy

Swiss Surgery ◽

10.1024/1023-9332.8.6.250 ◽

2002 ◽

Vol 8 (6) ◽

pp. 250-254

Author(s):

Vogelbach ◽

Bogdan ◽

Rosenthal ◽

Pfefferkorn ◽

Triponez

Keyword(s):

Virtual Reality ◽

Laparoscopic Cholecystectomy ◽

Neue Technik

Fragestellung: Die dieser Untersuchung zugrunde liegende Frage war, ob das angewandte Ausbildungskonzept geeignet war, um am Beispiel der Einführung der laparoskopischen Cholezystektomie eine neue Operationsmethode in einer universitären Ausbildungsklinik mit einer grossen Anzahl Chirurgen zu etablieren. Patienten und Methodik: Seit Einführung der ersten laparoskopischen Cholezystektomie wurden alle Cholezystektomien während zwei Jahren (Mai 1990 bis Mai 1992) prospektiv erfasst. Ein Ausbildungskonzept wurde gewählt, bei dem jeweils ein Operateur durch einen Tutor geschult wurde und so 15 konsekutive Eingriffe durchführte, um dann die Technik einem weiteren auszubildenden Chirurgen zu instruieren. Resultate: In zwei Jahren wurden 355 Patienten cholezystektomiert. 60% der Operationen wurden laparoskopisch durchgeführt oder begonnen. 40% der Operationen wurden offen durchgeführt. In den ersten zwei Jahren konnten 13 Operateure (durchschnittlich 16 Operationen / Operateur , range 1 - 60) in die neue Technik eingeführt werden. Es traten keine schweren Komplikationen, insbesondere keine Gallenwegsverletzungen in dieser Einführungsphase auf. Diskussion: In der Literatur wird dieses Vorgehen bei der Einführungsphase seit 1992 wiederholt vorgestellt, diskutiert und empfohlen. Zwischenzeitlich gibt es Richtlinien von Fachgesellschaften und nationalen Institutionen, welche die Ausbildung zur Ausführung neuer chirurgischer Techniken reglementieren. In den letzten Jahren verlagern sich die ersten Ausbildungsschritte in Richtung Trainingskurse an skill-stations und virtual reality Trainer. Schlussfolgerung: Das beschriebene Ausbildungskonzept bewährte sich in der Einführungsphase der laparoskopischen Cholezystektomie zu Beginn der 90er-Jahre.

Download Full-text

An Alternative Method for Identifying a Failed Engine in Twin-Engine Propeller Aircraft

Aviation Psychology and Applied Human Factors ◽

10.1027/2192-0923/a000195 ◽

2020 ◽

Vol 10 (2) ◽

pp. 103-111

Author(s):

Andrey K. Babin ◽

Andrew R. Dattel ◽

Margaret F. Klemm

Keyword(s):

Human Error ◽

Flight Simulator ◽

Traditional Group ◽

Aircraft Accidents ◽

Alternative Method ◽

Indicator Group ◽

Group A ◽

The Dead ◽

Engine Failure ◽

And Performance

Abstract. Twin-engine propeller aircraft accidents occur due to mechanical reasons as well as human error, such as misidentifying a failed engine. This paper proposes a visual indicator as an alternative method to the dead leg–dead engine procedure to identify a failed engine. In total, 50 pilots without a multi-engine rating were randomly assigned to a traditional (dead leg–dead engine) or an alternative (visual indicator) group. Participants performed three takeoffs in a flight simulator with a simulated engine failure after rotation. Participants in the alternative group identified the failed engine faster than the traditional group. A visual indicator may improve pilot accuracy and performance during engine-out emergencies and is recommended as a possible alternative for twin-engine propeller aircraft.

Download Full-text