3D Audio Perception System for Humanoid Robots

MOTION PLANNING USING PREDICTED PERCEPTIVE CAPABILITY

International Journal of Humanoid Robotics ◽

10.1142/s0219843609001826 ◽

2009 ◽

Vol 06 (03) ◽

pp. 435-457 ◽

Cited By ~ 5

Author(s):

PHILIPP MICHEL ◽

JOEL CHESTNUTT ◽

SATOSHI KAGAMI ◽

KOICHI NISHIWAKI ◽

JAMES J. KUFFNER ◽

...

Keyword(s):

Motion Planning ◽

Planning Process ◽

Humanoid Robots ◽

Experimental Results ◽

Success Rates ◽

Perception System ◽

Task Success

We present an approach to motion planning for humanoid robots that aims to ensure reliable execution by augmenting the planning process to reason about the robot's ability to successfully perceive its environment during operation. By efficiently simulating the robot's perception system during search, our planner utilizes a perceptive capability metric that quantifies the 'sensability' of the environment in each state given the task to be accomplished. We have applied our method to the problem of planning robust autonomous grasping motions and walking sequences as performed by an HRP-2 humanoid. A fast GPU-accelerated 3D tracker is used for perception, with a grasp planner and footstep planner incorporating reasoning about the robot's perceptive capability. Experimental results show that considering information about the predicted perceptive capability ensures that sensing remains operational throughout the grasping or walking sequence and yields higher task success rates than perception-unaware planning.

Download Full-text

Active Visual Perception for Humanoid Robots

International Journal of Humanoid Robotics ◽

10.1142/s0219843615500097 ◽

2015 ◽

Vol 12 (01) ◽

pp. 1550009 ◽

Cited By ~ 1

Author(s):

Francisco Martín ◽

Carlos E. Agüero ◽

José M. Cañas

Keyword(s):

Visual Perception ◽

Visual Attention ◽

Visual Memory ◽

Humanoid Robot ◽

Partial Information ◽

Internal Representation ◽

Humanoid Robots ◽

Software Components ◽

Perception System ◽

Mobile Cameras

Robots detect and keep track of relevant objects in their environment to accomplish some tasks. Many of them are equipped with mobile cameras as the main sensors, process the images and maintain an internal representation of the detected objects. We propose a novel active visual memory that moves the camera to detect objects in robot's surroundings and tracks their positions. This visual memory is based on a combination of multi-modal filters that efficiently integrates partial information. The visual attention subsystem is distributed among the software components in charge of detecting relevant objects. We demonstrate the efficiency and robustness of this perception system in a real humanoid robot participating in the RoboCup SPL competition.

Download Full-text

Emotion based human-robot interaction

MATEC Web of Conferences ◽

10.1051/matecconf/201816101001 ◽

2018 ◽

Vol 161 ◽

pp. 01001 ◽

Cited By ~ 3

Author(s):

Karsten Berns ◽

Zuhair Zafar

Keyword(s):

Humanoid Robot ◽

Humanoid Robots ◽

Verbal Communication ◽

Human Robot Interaction ◽

The Other ◽

Robot Interaction ◽

Human Machine Interaction ◽

Verbal Cues ◽

Perception System ◽

Machine Interaction

Human-machine interaction is a major challenge in the development of complex humanoid robots. In addition to verbal communication the use of non-verbal cues such as hand, arm and body gestures or mimics can improve the understanding of the intention of the robot. On the other hand, by perceiving such mechanisms of a human in a typical interaction scenario the humanoid robot can adapt its interaction skills in a better way. In this work, the perception system of two social robots, ROMAN and ROBIN of the RRLAB of the TU Kaiserslautern, is presented in the range of human-robot interaction.

Download Full-text

Usability of Individualized Head-Related Transfer Functions in Virtual Reality: Empirical Study With Perceptual Attributes in Sagittal Plane Sound Localization

JMIR Serious Games ◽

10.2196/17576 ◽

2020 ◽

Vol 8 (3) ◽

pp. e17576 ◽

Cited By ~ 1

Author(s):

Claudia Jenny ◽

Christoph Reuter

Keyword(s):

Virtual Reality ◽

Transfer Functions ◽

Sagittal Plane ◽

Rank Test ◽

Research Database ◽

Massachusetts Institute Of Technology ◽

3D Audio ◽

Localization Model ◽

Institute Of Technology ◽

Audio Perception

Background In order to present virtual sound sources via headphones spatially, head-related transfer functions (HRTFs) can be applied to audio signals. In this so-called binaural virtual acoustics, the spatial perception may be degraded if the HRTFs deviate from the true HRTFs of the listener. Objective In this study, participants wearing virtual reality (VR) headsets performed a listening test on the 3D audio perception of virtual audiovisual scenes, thus enabling us to investigate the necessity and influence of the individualization of HRTFs. Two hypotheses were investigated: first, general HRTFs lead to limitations of 3D audio perception in VR and second, the localization model for stationary localization errors is transferable to nonindividualized HRTFs in more complex environments such as VR. Methods For the evaluation, 39 subjects rated individualized and nonindividualized HRTFs in an audiovisual virtual scene on the basis of 5 perceptual qualities: localizability, front-back position, externalization, tone color, and realism. The VR listening experiment consisted of 2 tests: in the first test, subjects evaluated their own and the general HRTF from the Massachusetts Institute of Technology Knowles Electronics Manikin for Acoustic Research database and in the second test, their own and 2 other nonindividualized HRTFs from the Acoustics Research Institute HRTF database. For the experiment, 2 subject-specific, nonindividualized HRTFs with a minimal and maximal localization error deviation were selected according to the localization model in sagittal planes. Results With the Wilcoxon signed-rank test for the first test, analysis of variance for the second test, and a sample size of 78, the results were significant in all perceptual qualities, except for the front-back position between own and minimal deviant nonindividualized HRTF (P=.06). Conclusions Both hypotheses have been accepted. Sounds filtered by individualized HRTFs are considered easier to localize, easier to externalize, more natural in timbre, and thus more realistic compared to sounds filtered by nonindividualized HRTFs.

Download Full-text

Usability of Individualized Head-Related Transfer Functions in Virtual Reality: Empirical Study With Perceptual Attributes in Sagittal Plane Sound Localization (Preprint)

10.2196/preprints.17576 ◽

2019 ◽

Author(s):

Claudia Jenny ◽

Christoph Reuter

Keyword(s):

Virtual Reality ◽

Transfer Functions ◽

Sagittal Plane ◽

Rank Test ◽

Research Database ◽

Massachusetts Institute Of Technology ◽

3D Audio ◽

Localization Model ◽

Institute Of Technology ◽

Audio Perception

BACKGROUND In order to present virtual sound sources via headphones spatially, head-related transfer functions (HRTFs) can be applied to audio signals. In this so-called binaural virtual acoustics, the spatial perception may be degraded if the HRTFs deviate from the true HRTFs of the listener. OBJECTIVE In this study, participants wearing virtual reality (VR) headsets performed a listening test on the 3D audio perception of virtual audiovisual scenes, thus enabling us to investigate the necessity and influence of the individualization of HRTFs. Two hypotheses were investigated: first, general HRTFs lead to limitations of 3D audio perception in VR and second, the localization model for stationary localization errors is transferable to nonindividualized HRTFs in more complex environments such as VR. METHODS For the evaluation, 39 subjects rated individualized and nonindividualized HRTFs in an audiovisual virtual scene on the basis of 5 perceptual qualities: localizability, front-back position, externalization, tone color, and realism. The VR listening experiment consisted of 2 tests: in the first test, subjects evaluated their own and the general HRTF from the Massachusetts Institute of Technology Knowles Electronics Manikin for Acoustic Research database and in the second test, their own and 2 other nonindividualized HRTFs from the Acoustics Research Institute HRTF database. For the experiment, 2 subject-specific, nonindividualized HRTFs with a minimal and maximal localization error deviation were selected according to the localization model in sagittal planes. RESULTS With the Wilcoxon signed-rank test for the first test, analysis of variance for the second test, and a sample size of 78, the results were significant in all perceptual qualities, except for the front-back position between own and minimal deviant nonindividualized HRTF (<i>P</i>=.06). CONCLUSIONS Both hypotheses have been accepted. Sounds filtered by individualized HRTFs are considered easier to localize, easier to externalize, more natural in timbre, and thus more realistic compared to sounds filtered by nonindividualized HRTFs.

Download Full-text

BIOLOGICALLY BASED TOP-DOWN ATTENTION MODULATION FOR HUMANOID INTERACTIONS

International Journal of Humanoid Robotics ◽

10.1142/s0219843608001285 ◽

2008 ◽

Vol 05 (01) ◽

pp. 3-24 ◽

Cited By ~ 14

Author(s):

JAN MORÉN ◽

ALEŠ UDE ◽

ANSGAR KOENE ◽

GORDON CHENG

Keyword(s):

Visual Perception ◽

Humanoid Robots ◽

Dependent Manner ◽

Early Vision ◽

Top Down ◽

Bottom Up ◽

Low Level ◽

Perception System ◽

Meaningful Context ◽

Context Dependent

An adaptive perception system enables humanoid robots to interact with humans and their surroundings in a meaningful context-dependent manner. An important foundation for visual perception is the selectivity of early vision processes that enables the system to filter out low-level unimportant information while attending to features indicated as important by higher-level processes by way of top-down modulation. We present a novel way to integrate top-down and bottom-up processing for achieving such attention-based filtering. We specifically consider the case where the top-down target is not the most salient in any of the used submodalities.

Download Full-text