Wrapped Into Sound: Development of the Immersive Audio Quality Inventory (IAQI)

Although virtual reality, video entertainment, and computer games are dependent on the three-dimensional reproduction of sound (including front, rear, and height channels), it remains unclear whether 3D-audio formats actually intensify the emotional listening experience. There is currently no valid inventory for the objective measurement of immersive listening experiences resulting from audio playback formats with decreasing degrees of immersion (from mono to stereo, 5.1, and 3D). The development of the Immersive Audio Quality Inventory (IAQI) could close this gap. An initial item list (N = 25) was derived from studies in virtual reality and spatial audio, supplemented by researcher-developed items and items extracted from historical descriptions. Psychometric evaluation was conducted by an online study (N = 222 valid cases). Based on controlled headphone playback, participants listened to four songs/pieces, each in the three formats of mono, stereo, and binaural 3D audio. The latent construct “immersive listening experience” was determined by probabilistic test theory (item response theory, IRT) and by means of the many-facet Rasch measurement (MFRM). As a result, the specified MFRM model showed good model fit (62.69% of explained variance). The final one-dimensional inventory consists of 10 items and will be made available in English and German.

Immersive Audio Application Coding Proposal to the SBTVD TV 3.0 Call for Proposals

In July 2020 the Brazilian Terrestrial Television System Forum (SBTVD) has issued a Call for Proposals (CfP) for their next-generation digital TV system called TV 3.0. Fraunhofer IIS and ATEME have proposed the MPEG-H Audio system, based on the open international standard ISO/IEC 23008-3, MPEG-H 3D Audio, as a candidate technology for the Application Coding component of SBTVD TV 3.0. The submitted proposal specifies a new Application Programming Interface (API) enabling applications to make use of the nextgeneration interactivity features of the MPEG-H Audio system. This paper provides a detailed description of the proposed API, as well as the submitted JavaScript implementation, and the architecture of the prototype system. Additionally, the paper outlines the proposed evaluation process demonstrating how the MPEG-H Audio system fulfills the TV 3.0 Application Coding requirements for 3D object-based immersive audio interaction and emergency warning information delivery

Quantifying Auditory Presence Using Electroencephalography

Presence is used to assess the subjective experience of being in one place when physically situated in another. Recently, the research on presence has gained increasing attention due to the wide use of immersive audio technologies. Currently, the most widely-used measurement of presence is based on post-experiment self-report questionnaires. It is reliable but imperfect due to the psychological changes caused by the act of answering the questionnaire when immersed in the virtual environment. Therefore, the present work aims to find an objective way to measure presence, and electroencephalography (EEG) was investigated as a possible tool for this objective measurement. In this study, two listening tests were conducted, where eight loudspeakers were used to reproduce urban soundscapes to stimulate auditory presence. Presence was measured by both questionnaires and EEG. Results showed a significant correlation between T/B (Theta/Beta Ratio) extracted from EEG and subjective presence levels assessed by questionnaires, suggesting the possible use of EEG to measure presence objectively. This study could bring some insight for the research of presence, and related technologies, such as VR, video games and immersive audio production.

Immersive audio-visual scene reproduction using semantic scene reconstruction from 360 cameras

As personalised immersive display systems have been intensely explored in virtual reality (VR), plausible 3D audio corresponding to the visual content is required to provide more realistic experiences to users. It is well known that spatial audio synchronised with visual information improves a sense of immersion but limited research progress has been achieved in immersive audio-visual content production and reproduction. In this paper, we propose an end-to-end pipeline to simultaneously reconstruct 3D geometry and acoustic properties of the environment from a pair of omnidirectional panoramic images. A semantic scene reconstruction and completion method using a deep convolutional neural network is proposed to estimate the complete semantic scene geometry in order to adapt spatial audio reproduction to the scene. Experiments provide objective and subjective evaluations of the proposed pipeline for plausible audio-visual VR reproduction of real scenes.