scholarly journals Dynamic-DSO: Direct Sparse Odometry Using Objects Semantic Information for Dynamic Environments

2020 ◽  
Vol 10 (4) ◽  
pp. 1467
Author(s):  
Chao Sheng ◽  
Shuguo Pan ◽  
Wang Gao ◽  
Yong Tan ◽  
Tao Zhao
Keyword(s):  
Semantic Information ◽  
Direct Method ◽  
Visual Odometry ◽  
Dynamic Environments ◽  
Loop Closure ◽  
Loop Closure Detection ◽  
Camera Pose ◽  
Photometric Error ◽  
Tracking Model ◽  
Dynamic Objects

Traditional Simultaneous Localization and Mapping (SLAM) (with loop closure detection), or Visual Odometry (VO) (without loop closure detection), are based on the static environment assumption. When working in dynamic environments, they perform poorly whether using direct methods or indirect methods (feature points methods). In this paper, Dynamic-DSO which is a semantic monocular direct visual odometry based on DSO (Direct Sparse Odometry) is proposed. The proposed system is completely implemented with the direct method, which is different from the most current dynamic systems combining the indirect method with deep learning. Firstly, convolutional neural networks (CNNs) are applied to the original RGB image to generate the pixel-wise semantic information of dynamic objects. Then, based on the semantic information of the dynamic objects, dynamic candidate points are filtered out in keyframes candidate points extraction; only static candidate points are reserved in the tracking and optimization module, to achieve accurate camera pose estimation in dynamic environments. The photometric error calculated by the projection points in dynamic region of subsequent frames are removed from the whole photometric error in pyramid motion tracking model. Finally, the sliding window optimization which neglects the photometric error calculated in the dynamic region of each keyframe is applied to obtain the precise camera pose. Experiments on the public TUM dynamic dataset and the modified Euroc dataset show that the positioning accuracy and robustness of the proposed Dynamic-DSO is significantly higher than the state-of-the-art direct method in dynamic environments, and the semi-dense cloud map constructed by Dynamic-DSO is clearer and more detailed.

scholarly journals LIO-CSI: LiDAR inertial odometry with loop closure combined with semantic information

PLoS ONE ◽  
2021 ◽  
Vol 16 (12) ◽  
pp. e0261053
Author(s):  
Gang Wang ◽  
Saihang Gao ◽  
Han Ding ◽  
Hao Zhang ◽  
Hongmin Cai
Keyword(s):  
Point Cloud ◽  
Semantic Information ◽  
Feature Matching ◽  
Autonomous Driving ◽  
Geometric Feature ◽  
Semantic Features ◽  
Loop Closure ◽  
Loop Closure Detection ◽  
Front End ◽  
Dynamic Objects

Accurate and reliable state estimation and mapping are the foundation of most autonomous driving systems. In recent years, researchers have focused on pose estimation through geometric feature matching. However, most of the works in the literature assume a static scenario. Moreover, a registration based on a geometric feature is vulnerable to the interference of a dynamic object, resulting in a decline of accuracy. With the development of a deep semantic segmentation network, we can conveniently obtain the semantic information from the point cloud in addition to geometric information. Semantic features can be used as an accessory to geometric features that can improve the performance of odometry and loop closure detection. In a more realistic environment, semantic information can filter out dynamic objects in the data, such as pedestrians and vehicles, which lead to information redundancy in generated map and map-based localization failure. In this paper, we propose a method called LiDAR inertial odometry (LIO) with loop closure combined with semantic information (LIO-CSI), which integrates semantic information to facilitate the front-end process as well as loop closure detection. First, we made a local optimization on the semantic labels provided by the Sparse Point-Voxel Neural Architecture Search (SPVNAS) network. The optimized semantic information is combined into the front-end process of tightly-coupled light detection and ranging (LiDAR) inertial odometry via smoothing and mapping (LIO-SAM), which allows us to filter dynamic objects and improve the accuracy of the point cloud registration. Then, we proposed a semantic assisted scan-context method to improve the accuracy and robustness of loop closure detection. The experiments were conducted on an extensively used dataset KITTI and a self-collected dataset on the Jilin University (JLU) campus. The experimental results demonstrate that our method is better than the purely geometric method, especially in dynamic scenarios, and it has a good generalization ability.

Fast and robust visual odometry with a low-cost IMU in dynamic environments

2019 ◽  
Vol 46 (6) ◽  
pp. 882-894 ◽  
Author(s):  
Erliang Yao ◽  
Hexin Zhang ◽  
Haitao Song ◽  
Guoliang Zhang
Keyword(s):  
Indirect Method ◽  
Direct Method ◽  
Low Cost ◽  
Visual Odometry ◽  
Dynamic Environments ◽  
Motion Blur ◽  
Bundle Adjustment ◽  
Measurement Unit ◽  
Content Type ◽  
Camera Pose

Purpose To realize stable and precise localization in the dynamic environments, the authors propose a fast and robust visual odometry (VO) approach with a low-cost Inertial Measurement Unit (IMU) in this study. Design/methodology/approach The proposed VO incorporates the direct method with the indirect method to track the features and to optimize the camera pose. It initializes the positions of tracked pixels with the IMU information. Besides, the tracked pixels are refined by minimizing the photometric errors. Due to the small convergence radius of the indirect method, the dynamic pixels are rejected. Subsequently, the camera pose is optimized by minimizing the reprojection errors. The frames with little dynamic information are selected to create keyframes. Finally, the local bundle adjustment is performed to refine the poses of the keyframes and the positions of 3-D points. Findings The proposed VO approach is evaluated experimentally in dynamic environments with various motion types, suggesting that the proposed approach achieves more accurate and stable location than the conventional approach. Moreover, the proposed VO approach works well in the environments with the motion blur. Originality/value The proposed approach fuses the indirect method and the direct method with the IMU information, which improves the localization in dynamic environments significantly.

scholarly journals Robust Loop Closure Detection Integrating Visual–Spatial–Semantic Information via Topological Graphs and CNN Features

2020 ◽  
Vol 12 (23) ◽  
pp. 3890
Author(s):  
Yuwei Wang ◽  
Yuanying Qiu ◽  
Peitao Cheng ◽  
Xuechao Duan
Keyword(s):  
Semantic Information ◽  
Graph Matching ◽  
Spatial Relationships ◽  
Dynamic Scenes ◽  
Topological Graphs ◽  
Loop Closure ◽  
Loop Closure Detection ◽  
Visual Spatial ◽  
Localization And Mapping ◽  
The Impact

Loop closure detection is a key module for visual simultaneous localization and mapping (SLAM). Most previous methods for this module have not made full use of the information provided by images, i.e., they have only used the visual appearance or have only considered the spatial relationships of landmarks; the visual, spatial and semantic information have not been fully integrated. In this paper, a robust loop closure detection approach integrating visual–spatial–semantic information is proposed by employing topological graphs and convolutional neural network (CNN) features. Firstly, to reduce mismatches under different viewpoints, semantic topological graphs are introduced to encode the spatial relationships of landmarks, and random walk descriptors are employed to characterize the topological graphs for graph matching. Secondly, dynamic landmarks are eliminated by using semantic information, and distinctive landmarks are selected for loop closure detection, thus alleviating the impact of dynamic scenes. Finally, to ease the effect of appearance changes, the appearance-invariant descriptor of the landmark region is extracted by a pre-trained CNN without the specially designed manual features. The proposed approach weakens the influence of viewpoint changes and dynamic scenes, and extensive experiments conducted on open datasets and a mobile robot demonstrated that the proposed method has more satisfactory performance compared to state-of-the-art methods.

Loop Closure Detection for Visual SLAM Fusing Semantic Information

2019 ◽  
Author(s):  
Mingyue Hu ◽  
Sheng Li ◽  
Jingyuan Wu ◽  
Jiawei Guo ◽  
Haiyu Li ◽  
...  
Keyword(s):  
Semantic Information ◽  
Visual Slam ◽  
Loop Closure ◽  
Loop Closure Detection

scholarly journals Visual SLAM Framework Based on Segmentation with the Improvement of Loop Closure Detection in Dynamic Environments

2021 ◽  
Vol 33 (6) ◽  
pp. 1385-1397
Author(s):  
Leyuan Sun ◽  
Rohan P. Singh ◽  
Fumio Kanehiro ◽  
◽  
◽  
...  
Keyword(s):  
Point Cloud ◽  
Three Dimensional ◽  
Dynamic Environments ◽  
Camera Tracking ◽  
Loop Closure ◽  
Loop Closure Detection ◽  
Multiple Datasets ◽  
Localization And Mapping ◽  
Object Based ◽  
Precision Recall Curve

Most simultaneous localization and mapping (SLAM) systems assume that SLAM is conducted in a static environment. When SLAM is used in dynamic environments, the accuracy of each part of the SLAM system is adversely affected. We term this problem as dynamic SLAM. In this study, we propose solutions for three main problems in dynamic SLAM: camera tracking, three-dimensional map reconstruction, and loop closure detection. We propose to employ geometry-based method, deep learning-based method, and the combination of them for object segmentation. Using the information from segmentation to generate the mask, we filter the keypoints that lead to errors in visual odometry and features extracted by the CNN from dynamic areas to improve the performance of loop closure detection. Then, we validate our proposed loop closure detection method using the precision-recall curve and also confirm the framework’s performance using multiple datasets. The absolute trajectory error and relative pose error are used as metrics to evaluate the accuracy of the proposed SLAM framework in comparison with state-of-the-art methods. The findings of this study can potentially improve the robustness of SLAM technology in situations where mobile robots work together with humans, while the object-based point cloud byproduct has potential for other robotics tasks.

Modest-vocabulary loop-closure detection with incremental bag of tracked words

2021 ◽  
pp. 103782
Author(s):  
Konstantinos A. Tsintotas ◽  
Loukas Bampis ◽  
Antonios Gasteratos
Keyword(s):  
Loop Closure ◽  
Loop Closure Detection

scholarly journals Gaussian Process Gradient Maps for Loop-Closure Detection in Unstructured Planetary Environments

2020 ◽  
Author(s):  
Cedric Le Gentil ◽  
Mallikarjuna Vayugundla ◽  
Riccardo Giubilato ◽  
Wolfgang Sturzl ◽  
Teresa Vidal-Calleja ◽  
...  
Keyword(s):  
Gaussian Process ◽  
Loop Closure ◽  
Loop Closure Detection ◽  
Planetary Environments
Sign in / Sign up

Export Citation Format

Share Document