Kalman Filtering for Sensor Fusion in a Human Tracking System

Purpose This paper aims to develop a robust person tracking method for human following robots. The tracking system adopts the multimodal fusion results of millimeter wave (MMW) radars and monocular cameras for perception. A prototype of human following robot is developed and evaluated by using the proposed tracking system. Design/methodology/approach Limited by angular resolution, point clouds from MMW radars are too sparse to form features for human detection. Monocular cameras can provide semantic information for objects in view, but cannot provide spatial locations. Considering the complementarity of the two sensors, a sensor fusion algorithm based on multimodal data combination is proposed to identify and localize the target person under challenging conditions. In addition, a closed-loop controller is designed for the robot to follow the target person with expected distance. Findings A series of experiments under different circumstances are carried out to validate the fusion-based tracking method. Experimental results show that the average tracking errors are around 0.1 m. It is also found that the robot can handle different situations and overcome short-term interference, continually track and follow the target person. Originality/value This paper proposed a robust tracking system with the fusion of MMW radars and cameras. Interference such as occlusion and overlapping are well handled with the help of the velocity information from the radars. Compared to other state-of-the-art plans, the sensor fusion method is cost-effective and requires no additional tags with people. Its stable performance shows good application prospects in human following robots.

Download Full-text

Development and Experimental Verification of a Person Tracking System of Mobile Robots Using Sensor Fusion of Inertial Measurement Unit and Laser Range Finder for Occlusion Avoidance

Journal of Robotics and Mechatronics ◽

10.20965/jrm.2021.p0033 ◽

2021 ◽

Vol 33 (1) ◽

pp. 33-43

Author(s):

Kazuhiro Funato ◽

Ryosuke Tasaki ◽

Hiroto Sakurai ◽

Kazuhiko Terashima ◽

◽

...

Keyword(s):

Mobile Robot ◽

Sensor Fusion ◽

Experimental Verification ◽

Inertial Measurement Unit ◽

Tracking System ◽

Measurement Unit ◽

Laser Range Finder ◽

Range Finder ◽

Inertial Measurement ◽

Laser Range

The authors have been developing a mobile robot to assist doctors in hospitals in managing medical tools and patient electronic medical records. The robot tracks behind a mobile medical worker while maintaining a constant distance from the worker. However, it was difficult to detect objects in the sensor’s invisible region, called occlusion. In this study, we propose a sensor fusion method to estimate the position of a robot tracking target indirectly by an inertial measurement unit (IMU) in addition to the direct measurement by an laser range finder (LRF) and develop a human tracking system to avoid occlusion by a mobile robot. Based on this, we perform detailed experimental verification of tracking a specified person to verify the validity of the proposed method.

Download Full-text