scholarly journals Object Semantic Grid Mapping with 2D LiDAR and RGB-D Camera for Domestic Robot Navigation

2020 ◽  
Vol 10 (17) ◽  
pp. 5782 ◽  
Author(s):  
Xianyu Qi ◽  
Wei Wang ◽  
Ziwei Liao ◽  
Xiaoyu Zhang ◽  
Dongsheng Yang ◽  
...  
Keyword(s):  
Object Detection ◽  
Semantic Grid ◽  
Goal Selection ◽  
Depth Images ◽  
Robot Trajectory ◽  
Localization And Mapping ◽  
Detection Depth ◽  
Mapping System ◽  
Grid Mapping ◽  
Domestic Environments

Occupied grid maps are sufficient for mobile robots to complete metric navigation tasks in domestic environments. However, they lack semantic information to endow the robots with the ability of social goal selection and human-friendly operation modes. In this paper, we propose an object semantic grid mapping system with 2D Light Detection and Ranging (LiDAR) and RGB-D sensors to solve this problem. At first, we use a laser-based Simultaneous Localization and Mapping (SLAM) to generate an occupied grid map and obtain a robot trajectory. Then, we employ object detection to get an object’s semantics of color images and use joint interpolation to refine camera poses. Based on object detection, depth images, and interpolated poses, we build a point cloud with object instances. To generate object-oriented minimum bounding rectangles, we propose a method for extracting the dominant directions of the room. Furthermore, we build object goal spaces to help the robots select navigation goals conveniently and socially. We have used the Robot@Home dataset to verify the system; the verification results show that our system is effective.

scholarly journals Deep Learning-Based Object Detection, Localisation and Tracking for Smart Wheelchair Healthcare Mobility

2020 ◽  
Vol 18 (1) ◽  
pp. 91
Author(s):  
Louis Lecrosnier ◽  
Redouane Khemmar ◽  
Nicolas Ragot ◽  
Benoit Decoux ◽  
Romain Rossi ◽  
...  
Keyword(s):  
Object Detection ◽  
Object Tracking ◽  
Indoor Environment ◽  
Distance Estimation ◽  
Depth Estimation ◽  
Detection Algorithm ◽  
Use Case ◽  
Detection Distance ◽  
Detection Depth ◽  
Sort Algorithm

This paper deals with the development of an Advanced Driver Assistance System (ADAS) for a smart electric wheelchair in order to improve the autonomy of disabled people. Our use case, built from a formal clinical study, is based on the detection, depth estimation, localization and tracking of objects in wheelchair’s indoor environment, namely: door and door handles. The aim of this work is to provide a perception layer to the wheelchair, enabling this way the detection of these keypoints in its immediate surrounding, and constructing of a short lifespan semantic map. Firstly, we present an adaptation of the YOLOv3 object detection algorithm to our use case. Then, we present our depth estimation approach using an Intel RealSense camera. Finally, as a third and last step of our approach, we present our 3D object tracking approach based on the SORT algorithm. In order to validate all the developments, we have carried out different experiments in a controlled indoor environment. Detection, distance estimation and object tracking are experimented using our own dataset, which includes doors and door handles.

scholarly journals A 3D Object Detection Based on Multi-Modality Sensors of USV

2019 ◽  
Vol 9 (3) ◽  
pp. 535 ◽  
Author(s):  
Yingying Wu ◽  
Huacheng Qin ◽  
Tao Liu ◽  
Hao Liu ◽  
Zhiqiang Wei
Keyword(s):  
Object Detection ◽  
Marine Environment ◽  
Autonomous Navigation ◽  
3D Object ◽  
Detection Depth ◽  
Generation Network ◽  
Institute Of Technology ◽  
3D Object Detection ◽  
Complicated Surface ◽  
Fusion Detection

Unmanned Surface Vehicles (USVs) are commonly equipped with multi-modality sensors. Fully utilized sensors could improve object detection of USVs. This could further contribute to better autonomous navigation. The purpose of this paper is to solve the problems of 3D object detection of USVs in complicated marine environment. We propose a 3D object detection Depth Neural Network based on multi-modality data of USVs. This model includes a modified Proposal Generation Network and Deep Fusion Detection Network. The Proposal Generation Network improves feature extraction. Meanwhile, the Deep Fusion Detection Network enhances the fusion performance and can achieve more accurate results of object detection. The model was tested on both the KITTI 3D object detection dataset (A project of Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago) and a self-collected offshore dataset. The model shows excellent performance in a small memory condition. The results further prove that the method based on deep learning can give good accuracy in conditions of complicated surface in marine environment.

What localizes beneath: A metric multisensor localization and mapping system for autonomous underground mining vehicles

2020 ◽  
Vol 38 (1) ◽  
pp. 5-27 ◽  
Author(s):  
Adam Jacobson ◽  
Fan Zeng ◽  
David Smith ◽  
Nigel Boswell ◽  
Thierry Peynot ◽  
...  
Keyword(s):  
Underground Mining ◽  
Localization And Mapping ◽  
Mapping System ◽  
Mining Vehicles

scholarly journals The Accuracy Comparison of Three Simultaneous Localization and Mapping (SLAM)-Based Indoor Mapping Technologies

Sensors ◽  
2018 ◽  
Vol 18 (10) ◽  
pp. 3228 ◽  
Author(s):  
Yuwei Chen ◽  
Jian Tang ◽  
Changhui Jiang ◽  
Lingli Zhu ◽  
Matti Lehtomäki ◽  
...  
Keyword(s):  
Indoor Environment ◽  
Simultaneous Localization And Mapping ◽  
Point Clouds ◽  
Medium Size ◽  
Accuracy Evaluation ◽  
Depth Cameras ◽  
Localization And Mapping ◽  
Mapping System ◽  
Small Footprint ◽  
Mapping Systems

The growing interest and the market for indoor Location Based Service (LBS) have been drivers for a huge demand for building data and reconstructing and updating of indoor maps in recent years. The traditional static surveying and mapping methods can’t meet the requirements for accuracy, efficiency and productivity in a complicated indoor environment. Utilizing a Simultaneous Localization and Mapping (SLAM)-based mapping system with ranging and/or camera sensors providing point cloud data for the maps is an auspicious alternative to solve such challenges. There are various kinds of implementations with different sensors, for instance LiDAR, depth cameras, event cameras, etc. Due to the different budgets, the hardware investments and the accuracy requirements of indoor maps are diverse. However, limited studies on evaluation of these mapping systems are available to offer a guideline of appropriate hardware selection. In this paper we try to characterize them and provide some extensive references for SLAM or mapping system selection for different applications. Two different indoor scenes (a L shaped corridor and an open style library) were selected to review and compare three different mapping systems, namely: (1) a commercial Matterport system equipped with depth cameras; (2) SLAMMER: a high accuracy small footprint LiDAR with a fusion of hector-slam and graph-slam approaches; and (3) NAVIS: a low-cost large footprint LiDAR with Improved Maximum Likelihood Estimation (IMLE) algorithm developed by the Finnish Geospatial Research Institute (FGI). Firstly, an L shaped corridor (2nd floor of FGI) with approximately 80 m length was selected as the testing field for Matterport testing. Due to the lack of quantitative evaluation of Matterport indoor mapping performance, we attempted to characterize the pros and cons of the system by carrying out six field tests with different settings. The results showed that the mapping trajectory would influence the final mapping results and therefore, there was optimal Matterport configuration for better indoor mapping results. Secondly, a medium-size indoor environment (the FGI open library) was selected for evaluation of the mapping accuracy of these three indoor mapping technologies: SLAMMER, NAVIS and Matterport. Indoor referenced maps were collected with a small footprint Terrestrial Laser Scanner (TLS) and using spherical registration targets. The 2D indoor maps generated by these three mapping technologies were assessed by comparing them with the reference 2D map for accuracy evaluation; two feature selection methods were also utilized for the evaluation: interactive selection and minimum bounding rectangles (MBRs) selection. The mapping RMS errors of SLAMMER, NAVIS and Matterport were 2.0 cm, 3.9 cm and 4.4 cm, respectively, for the interactively selected features, and the corresponding values using MBR features were 1.7 cm, 3.2 cm and 4.7 cm. The corresponding detection rates for the feature points were 100%, 98.9%, 92.3% for the interactive selected features and 100%, 97.3% and 94.7% for the automated processing. The results indicated that the accuracy of all the evaluated systems could generate indoor map at centimeter-level, but also variation of the density and quality of collected point clouds determined the applicability of a system into a specific LBS.

scholarly journals Deep Learning Based Object Recognition Using Physically-Realistic Synthetic Depth Scenes

2019 ◽  
Vol 1 (3) ◽  
pp. 883-903 ◽  
Author(s):  
Daulet Baimukashev ◽  
Alikhan Zhilisbayev ◽  
Askat Kuzdeuov ◽  
Artemiy Oleinikov ◽  
Denis Fadeyev ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Object Recognition ◽  
Data Collection ◽  
Object Detection ◽  
Depth Image ◽  
Cluttered Environment ◽  
Depth Data ◽  
Depth Images ◽  
Image Dataset

Recognizing objects and estimating their poses have a wide range of application in robotics. For instance, to grasp objects, robots need the position and orientation of objects in 3D. The task becomes challenging in a cluttered environment with different types of objects. A popular approach to tackle this problem is to utilize a deep neural network for object recognition. However, deep learning-based object detection in cluttered environments requires a substantial amount of data. Collection of these data requires time and extensive human labor for manual labeling. In this study, our objective was the development and validation of a deep object recognition framework using a synthetic depth image dataset. We synthetically generated a depth image dataset of 22 objects randomly placed in a 0.5 m × 0.5 m × 0.1 m box, and automatically labeled all objects with an occlusion rate below 70%. Faster Region Convolutional Neural Network (R-CNN) architecture was adopted for training using a dataset of 800,000 synthetic depth images, and its performance was tested on a real-world depth image dataset consisting of 2000 samples. Deep object recognizer has 40.96% detection accuracy on the real depth images and 93.5% on the synthetic depth images. Training the deep learning model with noise-added synthetic images improves the recognition accuracy for real images to 46.3%. The object detection framework can be trained on synthetically generated depth data, and then employed for object recognition on the real depth data in a cluttered environment. Synthetic depth data-based deep object detection has the potential to substantially decrease the time and human effort required for the extensive data collection and labeling.

Object detection and mapping for service robot tasks

Robotica ◽  
2007 ◽  
Vol 25 (2) ◽  
pp. 175-187 ◽  
Author(s):  
Staffan Ekvall ◽  
Danica Kragic ◽  
Patric Jensfelt
Keyword(s):  
Object Detection ◽  
Recognition System ◽  
Detection Algorithm ◽  
Semantic Knowledge ◽  
Service Robot ◽  
Map Building ◽  
Extensive Evaluation ◽  
Localization And Mapping ◽  
Different Types ◽  
Robot Tasks

SUMMARYThe problem studied in this paper is a mobile robot that autonomously navigates in a domestic environment, builds a map as it moves along and localizes its position in it. In addition, the robot detects predefined objects, estimates their position in the environment and integrates this with the localization module to automatically put the objects in the generated map. Thus, we demonstrate one of the possible strategies for the integration of spatial and semantic knowledge in a service robot scenario where a simultaneous localization and mapping (SLAM) and object detection recognition system work in synergy to provide a richer representation of the environment than it would be possible with either of the methods alone. Most SLAM systems build maps that are only used for localizing the robot. Such maps are typically based on grids or different types of features such as point and lines. The novelty is the augmentation of this process with an object-recognition system that detects objects in the environment and puts them in the map generated by the SLAM system. The metric map is also split into topological entities corresponding to rooms. In this way, the user can command the robot to retrieve a certain object from a certain room. We present the results of map building and an extensive evaluation of the object detection algorithm performed in an indoor setting.

Sign in / Sign up

Export Citation Format

Share Document