scholarly journals SFPD: Simultaneous Face and Person Detection in Real-Time for Human–Robot Interaction

Sensors ◽  
2021 ◽  
Vol 21 (17) ◽  
pp. 5918
Author(s):  
Marc-André Fiedler ◽  
Philipp Werner ◽  
Aly Khalifa ◽  
Ayoub Al-Hamadi
Keyword(s):  
Real Time ◽  
Network Architecture ◽  
Human Action ◽  
Automatic Generation ◽  
Detection Performance ◽  
Human Robot Interaction ◽  
Training Procedure ◽  
Robot Interaction ◽  
Person Detection ◽  
Real Time Processing

Face and person detection are important tasks in computer vision, as they represent the first component in many recognition systems, such as face recognition, facial expression analysis, body pose estimation, face attribute detection, or human action recognition. Thereby, their detection rate and runtime are crucial for the performance of the overall system. In this paper, we combine both face and person detection in one framework with the goal of reaching a detection performance that is competitive to the state of the art of lightweight object-specific networks while maintaining real-time processing speed for both detection tasks together. In order to combine face and person detection in one network, we applied multi-task learning. The difficulty lies in the fact that no datasets are available that contain both face as well as person annotations. Since we did not have the resources to manually annotate the datasets, as it is very time-consuming and automatic generation of ground truths results in annotations of poor quality, we solve this issue algorithmically by applying a special training procedure and network architecture without the need of creating new labels. Our newly developed method called Simultaneous Face and Person Detection (SFPD) is able to detect persons and faces with 40 frames per second. Because of this good trade-off between detection performance and inference time, SFPD represents a useful and valuable real-time framework especially for a multitude of real-world applications such as, e.g., human–robot interaction.

scholarly journals Online control programming algorithm for human–robot interaction system with a novel real-time human gesture recognition method

2019 ◽  
Vol 16 (4) ◽  
pp. 172988141986176 ◽  
Author(s):  
Bo Chen ◽  
Chunsheng Hua ◽  
Bo Dai ◽  
Yuqing He ◽  
Jianda Han
Keyword(s):  
Real Time ◽  
Gesture Recognition ◽  
Robot Control ◽  
Recognition System ◽  
Online Control ◽  
Human Robot Interaction ◽  
Programming Algorithm ◽  
Recognition Method ◽  
Robot Interaction ◽  
Real Time Processing

This article proposes an online control programming algorithm for human–robot interaction systems, where robot actions are controlled by the recognition results of gestures performed by human operators based on visual images. In contrast to traditional robot control systems that use pre-defined programs to control a robot where the robot cannot change its tasks freely, this system allows the operator to train online and replan human–robot interaction tasks in real time. The proposed system is comprised of three components: an online personal feature pretraining system, a gesture recognition system, and a task replanning system for robot control. First, we collected and analyzed features extracted from images of human gestures and used those features to train the recognition program in real time. Second, a multifeature cascade classifier algorithm was applied to guarantee both the accuracy and real-time processing of our gesture recognition method. Finally, to confirm the effectiveness of our algorithm, we selected a flight robot as our test platform to conduct an online robot control experiment based on the visual gesture recognition algorithm. Through extensive experiments, the effectiveness and efficiency of our method has been confirmed.

scholarly journals Real-Time Instance Segmentation of Traffic Videos for Embedded Devices

Sensors ◽  
2021 ◽  
Vol 21 (1) ◽  
pp. 275
Author(s):  
Ruben Panero Martinez ◽  
Ionut Schiopu ◽  
Bruno Cornelis ◽  
Adrian Munteanu
Keyword(s):  
Real Time ◽  
Network Architecture ◽  
Training Procedure ◽  
Segmentation Method ◽  
Embedded Devices ◽  
Network Training ◽  
Assignment Algorithm ◽  
Ablation Study ◽  
Reduced Rate ◽  
Instance Segmentation

The paper proposes a novel instance segmentation method for traffic videos devised for deployment on real-time embedded devices. A novel neural network architecture is proposed using a multi-resolution feature extraction backbone and improved network designs for the object detection and instance segmentation branches. A novel post-processing method is introduced to ensure a reduced rate of false detection by evaluating the quality of the output masks. An improved network training procedure is proposed based on a novel label assignment algorithm. An ablation study on speed-vs.-performance trade-off further modifies the two branches and replaces the conventional ResNet-based performance-oriented backbone with a lightweight speed-oriented design. The proposed architectural variations achieve real-time performance when deployed on embedded devices. The experimental results demonstrate that the proposed instance segmentation method for traffic videos outperforms the you only look at coefficients algorithm, the state-of-the-art real-time instance segmentation method. The proposed architecture achieves qualitative results with 31.57 average precision on the COCO dataset, while its speed-oriented variations achieve speeds of up to 66.25 frames per second on the Jetson AGX Xavier module.

Human-robot interaction in industry 4.0 based on an internet of things real-time gesture control system

2020 ◽  
pp. 1-17
Author(s):  
Luis Roda-Sanchez ◽  
Teresa Olivares ◽  
Celia Garrido-Hidalgo ◽  
José Luis de la Vara ◽  
Antonio Fernández-Caballero
Keyword(s):  
Control System ◽  
Internet Of Things ◽  
Real Time ◽  
Industry 4.0 ◽  
Human Robot Interaction ◽  
Robot Interaction ◽  
Gesture Control
Sign in / Sign up

Export Citation Format

Share Document