scholarly journals The Method of Static Semantic Map Construction Based on Instance Segmentation and Dynamic Point Elimination

Electronics ◽  
2021 ◽  
Vol 10 (16) ◽  
pp. 1883
Author(s):  
Jingyu Li ◽  
Rongfen Zhang ◽  
Yuhong Liu ◽  
Zaiteng Zhang ◽  
Runze Fan ◽  
...  
Keyword(s):  
Semantic Information ◽  
Semantic Map ◽  
Average Value ◽  
Map Construction ◽  
Dense Optical Flow ◽  
Semantic Maps ◽  
Localization And Mapping ◽  
Dynamic Objects ◽  
High Level ◽  
Static Semantic

Semantic information usually contains a description of the environment content, which enables mobile robot to understand the environment and improves its ability to interact with the environment. In high-level human–computer interaction application, the Simultaneous Localization and Mapping (SLAM) system not only needs higher accuracy and robustness, but also has the ability to construct a static semantic map of the environment. However, traditional visual SLAM lacks semantic information. Furthermore, in an actual scene, dynamic objects will reduce the system performance and also generate redundancy when constructing map. these all directly affect the robot’s ability to perceive and understand the surrounding environment. Based on ORB-SLAM3, this article proposes a new Algorithm that uses semantic information and the global dense optical flow as constraints to generate dynamic-static mask and eliminate dynamic objects. then, to further construct a static 3D semantic map under indoor dynamic environments, a fusion of 2D semantic information and 3D point cloud is carried out. the experimental results on different types of dataset sequences show that, compared with original ORB-SLAM3, both Absolute Pose Error (APE) and Relative Pose Error (RPE) have been ameliorated to varying degrees, especially on freiburg3-walking-xyz, the APE reduced by 97.78% from the original average value of 0.523, and RPE reduced by 52.33% from the original average value of 0.0193. Compared with DS-SLAM and DynaSLAM, our system improves real-time performance while ensuring accuracy and robustness. Meanwhile, the expected map with environmental semantic information is built, and the map redundancy caused by dynamic objects is successfully reduced. the test results in real scenes further demonstrate the effect of constructing static semantic maps and prove the effectiveness of our Algorithm.

scholarly journals Semantic Mapping for Mobile Robots in Indoor Scenes: A Survey

Information ◽  
2021 ◽  
Vol 12 (2) ◽  
pp. 92
Author(s):  
Xiaoning Han ◽  
Shuailong Li ◽  
Xiaohui Wang ◽  
Weijia Zhou
Keyword(s):  
Mobile Robots ◽  
Semantic Information ◽  
Semantic Mapping ◽  
Human Beings ◽  
Future Directions ◽  
Essential Requirement ◽  
Semantic Map ◽  
Semantic Maps ◽  
Great Progress ◽  
Indoor Scenes

Sensing and mapping its surroundings is an essential requirement for a mobile robot. Geometric maps endow robots with the capacity of basic tasks, e.g., navigation. To co-exist with human beings in indoor scenes, the need to attach semantic information to a geometric map, which is called a semantic map, has been realized in the last two decades. A semantic map can help robots to behave in human rules, plan and perform advanced tasks, and communicate with humans on the conceptual level. This survey reviews methods about semantic mapping in indoor scenes. To begin with, we answered the question, what is a semantic map for mobile robots, by its definitions. After that, we reviewed works about each of the three modules of semantic mapping, i.e., spatial mapping, acquisition of semantic information, and map representation, respectively. Finally, though great progress has been made, there is a long way to implement semantic maps in advanced tasks for robots, thus challenges and potential future directions are discussed before a conclusion at last.

scholarly journals SeDAR: Reading Floorplans Like a Human—Using Deep Learning to Enable Human-Inspired Localisation

2019 ◽  
Vol 128 (5) ◽  
pp. 1286-1310 ◽  
Author(s):  
Oscar Mendez ◽  
Simon Hadfield ◽  
Nicolas Pugeault ◽  
Richard Bowden
Keyword(s):  
Deep Learning ◽  
Semantic Information ◽  
State Of The Art ◽  
Depth Information ◽  
Semantic Maps ◽  
Novel Method ◽  
Rgb Images ◽  
High Level ◽  
Robotic Tasks ◽  
And Robotics

Abstract The use of human-level semantic information to aid robotic tasks has recently become an important area for both Computer Vision and Robotics. This has been enabled by advances in Deep Learning that allow consistent and robust semantic understanding. Leveraging this semantic vision of the world has allowed human-level understanding to naturally emerge from many different approaches. Particularly, the use of semantic information to aid in localisation and reconstruction has been at the forefront of both fields. Like robots, humans also require the ability to localise within a structure. To aid this, humans have designed high-level semantic maps of our structures called floorplans. We are extremely good at localising in them, even with limited access to the depth information used by robots. This is because we focus on the distribution of semantic elements, rather than geometric ones. Evidence of this is that humans are normally able to localise in a floorplan that has not been scaled properly. In order to grant this ability to robots, it is necessary to use localisation approaches that leverage the same semantic information humans use. In this paper, we present a novel method for semantically enabled global localisation. Our approach relies on the semantic labels present in the floorplan. Deep Learning is leveraged to extract semantic labels from RGB images, which are compared to the floorplan for localisation. While our approach is able to use range measurements if available, we demonstrate that they are unnecessary as we can achieve results comparable to state-of-the-art without them.

Robust Semantic Mapping in Challenging Environments

Robotica ◽  
2019 ◽  
Vol 38 (2) ◽  
pp. 256-270 ◽  
Author(s):  
Jiyu Cheng ◽  
Yuxiang Sun ◽  
Max Q.-H. Meng
Keyword(s):  
Semantic Information ◽  
Moving Objects ◽  
Dynamic Environments ◽  
Semantic Mapping ◽  
Visual Slam ◽  
The Public ◽  
Semantic Map ◽  
Recent Success ◽  
3D Space ◽  
Localization And Mapping

SummaryVisual simultaneous localization and mapping (visual SLAM) has been well developed in recent decades. To facilitate tasks such as path planning and exploration, traditional visual SLAM systems usually provide mobile robots with the geometric map, which overlooks the semantic information. To address this problem, inspired by the recent success of the deep neural network, we combine it with the visual SLAM system to conduct semantic mapping. Both the geometric and semantic information will be projected into the 3D space for generating a 3D semantic map. We also use an optical-flow-based method to deal with the moving objects such that our method is capable of working robustly in dynamic environments. We have performed our experiments in the public TUM dataset and our recorded office dataset. Experimental results demonstrate the feasibility and impressive performance of the proposed method.

scholarly journals An Improved Deep Residual Network-Based Semantic Simultaneous Localization and Mapping Method for Monocular Vision Robot

2020 ◽  
Vol 2020 ◽  
pp. 1-14 ◽  
Author(s):  
Jianjun Ni ◽  
Tao Gong ◽  
Yafei Gu ◽  
Jinxiu Zhu ◽  
Xinnan Fan
Keyword(s):  
Semantic Information ◽  
Hot Spot ◽  
Simultaneous Localization And Mapping ◽  
Mapping Method ◽  
Monocular Vision ◽  
Visual Slam ◽  
Camera Tracking ◽  
Residual Network ◽  
Semantic Map ◽  
Localization And Mapping

The robot simultaneous localization and mapping (SLAM) is a very important and useful technology in the robotic field. However, the environmental map constructed by the traditional visual SLAM method contains little semantic information, which cannot satisfy the needs of complex applications. The semantic map can deal with this problem efficiently, which has become a research hot spot. This paper proposed an improved deep residual network- (ResNet-) based semantic SLAM method for monocular vision robots. In the proposed approach, an improved image matching algorithm based on feature points is presented, to enhance the anti-interference ability of the algorithm. Then, the robust feature point extraction method is adopted in the front-end module of the SLAM system, which can effectively reduce the probability of camera tracking loss. In addition, the improved key frame insertion method is introduced in the visual SLAM system to enhance the stability of the system during the turning and moving of the robot. Furthermore, an improved ResNet model is proposed to extract the semantic information of the environment to complete the construction of the semantic map of the environment. Finally, various experiments are conducted and the results show that the proposed method is effective.

scholarly journals Semantic Information for Robot Navigation: A Survey

2020 ◽  
Vol 10 (2) ◽  
pp. 497 ◽  
Author(s):  
Jonathan Crespo ◽  
Jose Carlos Castillo ◽  
Oscar Martinez Mozos ◽  
Ramon Barber
Keyword(s):  
Semantic Information ◽  
Robot Navigation ◽  
Semantic Knowledge ◽  
Early Years ◽  
Navigation Systems ◽  
Knowledge Representations ◽  
Semantic Maps ◽  
New Concepts ◽  
One Step ◽  
High Level

There is a growing trend in robotics for implementing behavioural mechanisms based on human psychology, such as the processes associated with thinking. Semantic knowledge has opened new paths in robot navigation, allowing a higher level of abstraction in the representation of information. In contrast with the early years, when navigation relied on geometric navigators that interpreted the environment as a series of accessible areas or later developments that led to the use of graph theory, semantic information has moved robot navigation one step further. This work presents a survey on the concepts, methodologies and techniques that allow including semantic information in robot navigation systems. The techniques involved have to deal with a range of tasks from modelling the environment and building a semantic map, to including methods to learn new concepts and the representation of the knowledge acquired, in many cases through interaction with users. As understanding the environment is essential to achieve high-level navigation, this paper reviews techniques for acquisition of semantic information, paying attention to the two main groups: human-assisted and autonomous techniques. Some state-of-the-art semantic knowledge representations are also studied, including ontologies, cognitive maps and semantic maps. All of this leads to a recent concept, semantic navigation, which integrates the previous topics to generate high-level navigation systems able to deal with real-world complex situations.

scholarly journals Online semantic mapping of logistic environments using RGB-D cameras

2017 ◽  
Vol 14 (4) ◽  
pp. 172988141772078 ◽  
Author(s):  
Marian Himstedt ◽  
Erik Maehle
Keyword(s):  
Recognition System ◽  
Automated Guided Vehicles ◽  
Semantic Mapping ◽  
The Novel ◽  
Robotic Navigation ◽  
Safe Navigation ◽  
Semantic Maps ◽  
Localization And Mapping ◽  
Geometric Representations ◽  
High Level

Automated guided vehicles require spatial representations of their working spaces in order to ensure safe navigation and carry out high-level tasks. Typically, these models are given by geometric maps. Even though these enable basic robotic navigation, they off-the-shelf lack the availability of task-dependent information required to provide services. This article presents a semantic mapping approach augmenting existing geometric representations. Our approach demonstrates the automatic annotation of map subspaces on the example of warehouse environments. The proposals of an object recognition system are integrated in a graph-based simultaneous localization and mapping framework and eventually propagated into a global map representation. Our system is experimentally evaluated in a typical warehouse consisting of common object classes expected for this type of environment. We discuss the novel achievements and motivate the contribution of semantic maps toward the operation of automated guided vehicles in the context of Industry 4.0.

scholarly journals LiDAR Odometry and Mapping Based on Semantic Information for Outdoor Environment

2021 ◽  
Vol 13 (15) ◽  
pp. 2864
Author(s):  
Shitong Du ◽  
Yifan Li ◽  
Xuyou Li ◽  
Menghao Wu
Keyword(s):  
Feature Extraction ◽  
Semantic Information ◽  
State Of The Art ◽  
Outdoor Environment ◽  
Geometric Features ◽  
Dynamic Object ◽  
Object Removal ◽  
Localization And Mapping ◽  
High Level ◽  
Crucial Part

Simultaneous Localization and Mapping (SLAM) in an unknown environment is a crucial part for intelligent mobile robots to achieve high-level navigation and interaction tasks. As one of the typical LiDAR-based SLAM algorithms, the Lidar Odometry and Mapping in Real-time (LOAM) algorithm has shown impressive results. However, LOAM only uses low-level geometric features without considering semantic information. Moreover, the lack of a dynamic object removal strategy limits the algorithm to obtain higher accuracy. To this end, this paper extends the LOAM pipeline by integrating semantic information into the original framework. Specifically, we first propose a two-step dynamic objects filtering strategy. Point-wise semantic labels are then used to improve feature extraction and searching for corresponding points. We evaluate the performance of the proposed method in many challenging scenarios, including highway, country and urban from the KITTI dataset. The results demonstrate that the proposed SLAM system outperforms the state-of-the-art SLAM methods in terms of accuracy and robustness.

scholarly journals A Computationally Efficient Semantic SLAM Solution for Dynamic Scenes

2019 ◽  
Vol 11 (11) ◽  
pp. 1363 ◽  
Author(s):  
Zemin Wang ◽  
Qian Zhang ◽  
Jiansheng Li ◽  
Shuming Zhang ◽  
Jingbin Liu
Keyword(s):  
Intelligent Systems ◽  
Semantic Information ◽  
Moving Objects ◽  
Location Based Services ◽  
Tracking Accuracy ◽  
Computationally Efficient ◽  
Dynamic Scenes ◽  
Localization Accuracy ◽  
Semantic Maps ◽  
Dynamic Objects

In various dynamic scenes, there are moveable objects such as pedestrians, which may challenge simultaneous localization and mapping (SLAM) algorithms. Consequently, the localization accuracy may be degraded, and a moving object may negatively impact the constructed maps. Maps that contain semantic information of dynamic objects impart humans or robots with the ability to semantically understand the environment, and they are critical for various intelligent systems and location-based services. In this study, we developed a computationally efficient SLAM solution that is able to accomplish three tasks in real time: (1) complete localization without accuracy loss due to the existence of dynamic objects and generate a static map that does not contain moving objects, (2) extract semantic information of dynamic objects through a computionally efficient approach, and (3) eventually generate semantic maps, which overlay semantic objects on static maps. The proposed semantic SLAM solution was evaluated through four different experiments on two data sets, respectively verifying the tracking accuracy, computational efficiency, and the quality of the generated static maps and semantic maps. The results show that the proposed SLAM solution is computationally efficient by reducing the time consumption for building maps by 2/3; moreover, the relative localization accuracy is improved, with a translational error of only 0.028 m, and is not degraded by dynamic objects. Finally, the proposed solution generates static maps of a dynamic scene without moving objects and semantic maps with high-precision semantic information of specific objects.

scholarly journals A survey: which features are required for dynamic visual simultaneous localization and mapping?

2021 ◽  
Vol 4 (1) ◽  
Author(s):  
Zewen Xu ◽  
Zheng Rong ◽  
Yihong Wu
Keyword(s):  
Simultaneous Localization And Mapping ◽  
Dynamic Environments ◽  
Visual Features ◽  
Advantages And Disadvantages ◽  
Intelligent Robots ◽  
Localization And Mapping ◽  
High Level ◽  
Static World ◽  
Robotic Applications ◽  
Significant Attention

AbstractIn recent years, simultaneous localization and mapping in dynamic environments (dynamic SLAM) has attracted significant attention from both academia and industry. Some pioneering work on this technique has expanded the potential of robotic applications. Compared to standard SLAM under the static world assumption, dynamic SLAM divides features into static and dynamic categories and leverages each type of feature properly. Therefore, dynamic SLAM can provide more robust localization for intelligent robots that operate in complex dynamic environments. Additionally, to meet the demands of some high-level tasks, dynamic SLAM can be integrated with multiple object tracking. This article presents a survey on dynamic SLAM from the perspective of feature choices. A discussion of the advantages and disadvantages of different visual features is provided in this article.

scholarly journals A Machine Learning Method for Vision-Based Unmanned Aerial Vehicle Systems to Understand Unknown Environments

Sensors ◽  
2020 ◽  
Vol 20 (11) ◽  
pp. 3245
Author(s):  
Tianyao Zhang ◽  
Xiaoguang Hu ◽  
Jin Xiao ◽  
Guofeng Zhang
Keyword(s):  
Machine Learning ◽  
Visual Servoing ◽  
Detection System ◽  
Detection Accuracy ◽  
Unknown Environment ◽  
Unknown Environments ◽  
Localization And Mapping ◽  
Computational Speed ◽  
Aerial Vehicle ◽  
High Level

What makes unmanned aerial vehicles (UAVs) intelligent is their capability of sensing and understanding new unknown environments. Some studies utilize computer vision algorithms like Visual Simultaneous Localization and Mapping (VSLAM) and Visual Odometry (VO) to sense the environment for pose estimation, obstacles avoidance and visual servoing. However, understanding the new environment (i.e., make the UAV recognize generic objects) is still an essential scientific problem that lacks a solution. Therefore, this paper takes a step to understand the items in an unknown environment. The aim of this research is to enable the UAV with basic understanding capability for a high-level UAV flock application in the future. Specially, firstly, the proposed understanding method combines machine learning and traditional algorithm to understand the unknown environment through RGB images; secondly, the You Only Look Once (YOLO) object detection system is integrated (based on TensorFlow) in a smartphone to perceive the position and category of 80 classes of objects in the images; thirdly, the method makes the UAV more intelligent and liberates the operator from labor; fourthly, detection accuracy and latency in working condition are quantitatively evaluated, and properties of generality (can be used in various platforms), transportability (easily deployed from one platform to another) and scalability (easily updated and maintained) for UAV flocks are qualitatively discussed. The experiments suggest that the method has enough accuracy to recognize various objects with high computational speed, and excellent properties of generality, transportability and scalability.

Sign in / Sign up

Export Citation Format

Share Document