scholarly journals A Precise and Robust Segmentation-Based Lidar Localization System for Automated Urban Driving

2019 ◽  
Vol 11 (11) ◽  
pp. 1348 ◽  
Author(s):  
Hang Liu ◽  
Qin Ye ◽  
Hairui Wang ◽  
Liang Chen ◽  
Jian Yang
Keyword(s):  
Real Time ◽  
Semantic Segmentation ◽  
Autonomous Driving ◽  
Unmanned Vehicles ◽  
Measurement Unit ◽  
Integrated Navigation ◽  
Current Frame ◽  
Localization System ◽  
Robust Segmentation ◽  
Localization Information

Real-time and high-precision localization information is vital for many modules of unmanned vehicles. At present, a high-cost RTK (Real Time Kinematic) and IMU (Integrated Measurement Unit) integrated navigation system is often used, but its accuracy cannot meet the requirements and even fails in many scenes. In order to reduce the costs and improve the localization accuracy and stability, we propose a precise and robust segmentation-based Lidar (Light Detection and Ranging) localization system aided with MEMS (Micro-Electro-Mechanical System) IMU and designed for high level autonomous driving. Firstly, we extracted features from the online frame using a series of proposed efficient low-level semantic segmentation-based multiple types feature extraction algorithms, including ground, road-curb, edge, and surface. Next, we matched the adjacent frames in Lidar odometry module and matched the current frame with the dynamically loaded pre-build feature point cloud map in Lidar localization module based on the extracted features to precisely estimate the 6DoF (Degree of Freedom) pose, through the proposed priori information considered category matching algorithm and multi-group-step L-M (Levenberg-Marquardt) optimization algorithm. Finally, the lidar localization results were fused with MEMS IMU data through a state-error Kalman filter to produce smoother and more accurate localization information at a high frequency of 200Hz. The proposed localization system can achieve 3~5 cm in position and 0.05~0.1° in orientation RMS (Root Mean Square) accuracy and outperform previous state-of-the-art systems. The robustness and adaptability have been verified with localization testing data more than 1000 Km in various challenging scenes, including congested urban roads, narrow tunnels, textureless highways, and rain-like harsh weather.

scholarly journals Real-Time Semantic Segmentation with Dual Encoder and Self-Attention Mechanism for Autonomous Driving

Sensors ◽  
2021 ◽  
Vol 21 (23) ◽  
pp. 8072
Author(s):  
Yu-Bang Chang ◽  
Chieh Tsai ◽  
Chang-Hong Lin ◽  
Poki Chen
Keyword(s):  
Deep Learning ◽  
Real Time ◽  
Network Architecture ◽  
Semantic Segmentation ◽  
Autonomous Driving ◽  
Attention Mechanism ◽  
Trade Off ◽  
Segmentation Methods ◽  
General Semantic ◽  
Deep Learning Model

As the techniques of autonomous driving become increasingly valued and universal, real-time semantic segmentation has become very popular and challenging in the field of deep learning and computer vision in recent years. However, in order to apply the deep learning model to edge devices accompanying sensors on vehicles, we need to design a structure that has the best trade-off between accuracy and inference time. In previous works, several methods sacrificed accuracy to obtain a faster inference time, while others aimed to find the best accuracy under the condition of real time. Nevertheless, the accuracies of previous real-time semantic segmentation methods still have a large gap compared to general semantic segmentation methods. As a result, we propose a network architecture based on a dual encoder and a self-attention mechanism. Compared with preceding works, we achieved a 78.6% mIoU with a speed of 39.4 FPS with a 1024 × 2048 resolution on a Cityscapes test submission.

scholarly journals Real-Time LiDAR Point Cloud Semantic Segmentation for Autonomous Driving

Electronics ◽  
2021 ◽  
Vol 11 (1) ◽  
pp. 11
Author(s):  
Xing Xie ◽  
Lin Bai ◽  
Xinming Huang
Keyword(s):  
Real Time ◽  
Power Efficiency ◽  
Point Cloud ◽  
Processing Time ◽  
State Of The Art ◽  
Semantic Segmentation ◽  
Autonomous Driving ◽  
Geometric Information ◽  
Embedded Platform ◽  
Gpu Implementation

LiDAR has been widely used in autonomous driving systems to provide high-precision 3D geometric information about the vehicle’s surroundings for perception, localization, and path planning. LiDAR-based point cloud semantic segmentation is an important task with a critical real-time requirement. However, most of the existing convolutional neural network (CNN) models for 3D point cloud semantic segmentation are very complex and can hardly be processed at real-time on an embedded platform. In this study, a lightweight CNN structure was proposed for projection-based LiDAR point cloud semantic segmentation with only 1.9 M parameters that gave an 87% reduction comparing to the state-of-the-art networks. When evaluated on a GPU, the processing time was 38.5 ms per frame, and it achieved a 47.9% mIoU score on Semantic-KITTI dataset. In addition, the proposed CNN is targeted on an FPGA using an NVDLA architecture, which results in a 2.74x speedup over the GPU implementation with a 46 times improvement in terms of power efficiency.

scholarly journals Point Cloud Semantic Segmentation with Cross-Correction Features

2022 ◽  
Author(s):  
Yuehua Zhao ◽  
Ma Jie ◽  
Chong Nannan ◽  
Wen Junjie
Keyword(s):  
Real Time ◽  
Point Cloud ◽  
Large Scale ◽  
Spatial Information ◽  
Semantic Segmentation ◽  
Autonomous Driving ◽  
Basic Unit ◽  
Semantic Features ◽  
Point Cloud Segmentation ◽  
Scale Point

Abstract Real time large scale point cloud segmentation is an important but challenging task for practical application like autonomous driving. Existing real time methods have achieved acceptance performance by aggregating local information. However, most of them only exploit local spatial information or local semantic information dependently, few considering the complementarity of both. In this paper, we propose a model named Spatial-Semantic Incorporation Network (SSI-Net) for real time large scale point cloud segmentation. A Spatial-Semantic Cross-correction (SSC) module is introduced in SSI-Net as a basic unit. High quality contextual features can be learned through SSC by correct and update semantic features using spatial cues, and vice verse. Adopting the plug-and-play SSC module, we design SSI-Net as an encoder-decoder architecture. To ensure efficiency, it also adopts a random sample based hierarchical network structure. Extensive experiments on several prevalent datasets demonstrate that our method can achieve state-of-the-art performance.

A Comparative Study of Real-Time Semantic Segmentation for Autonomous Driving

2018 ◽  
Author(s):  
Mennatullah Siam ◽  
Mostafa Gamal ◽  
Moemen Abdel-Razek ◽  
Senthil Yogamani ◽  
Martin Jagersand ◽  
...  
Keyword(s):  
Comparative Study ◽  
Real Time ◽  
Semantic Segmentation ◽  
Autonomous Driving

scholarly journals Implementation of a Lightweight Semantic Segmentation Algorithm in Road Obstacle Detection

Sensors ◽  
2020 ◽  
Vol 20 (24) ◽  
pp. 7089
Author(s):  
Bushi Liu ◽  
Yongbo Lv ◽  
Yang Gu ◽  
Wanjun Lv
Keyword(s):  
Real Time ◽  
Spatial Information ◽  
Feature Fusion ◽  
Semantic Segmentation ◽  
Spatial Location ◽  
Autonomous Driving ◽  
Obstacle Detection ◽  
Depth Information ◽  
Long Time ◽  
Deep Learning Network

Due to deep learning’s accurate cognition of the street environment, the convolutional neural network has achieved dramatic development in the application of street scenes. Considering the needs of autonomous driving and assisted driving, in a general way, computer vision technology is used to find obstacles to avoid collisions, which has made semantic segmentation a research priority in recent years. However, semantic segmentation has been constantly facing new challenges for quite a long time. Complex network depth information, large datasets, real-time requirements, etc., are typical problems that need to be solved urgently in the realization of autonomous driving technology. In order to address these problems, we propose an improved lightweight real-time semantic segmentation network, which is based on an efficient image cascading network (ICNet) architecture, using multi-scale branches and a cascaded feature fusion unit to extract rich multi-level features. In this paper, a spatial information network is designed to transmit more prior knowledge of spatial location and edge information. During the course of the training phase, we append an external loss function to enhance the learning process of the deep learning network system as well. This lightweight network can quickly perceive obstacles and detect roads in the drivable area from images to satisfy autonomous driving characteristics. The proposed model shows substantial performance on the Cityscapes dataset. With the premise of ensuring real-time performance, several sets of experimental comparisons illustrate that SP-ICNet enhances the accuracy of road obstacle detection and provides nearly ideal prediction outputs. Compared to the current popular semantic segmentation network, this study also demonstrates the effectiveness of our lightweight network for road obstacle detection in autonomous driving.

scholarly journals DNS: A multi-scale deconvolution semantic segmentation network for joint detection and segmentation

2019 ◽  
Vol 277 ◽  
pp. 02005
Author(s):  
Ning Feng ◽  
Le Dong ◽  
Qianni Zhang ◽  
Ning Zhang ◽  
Xi Wu ◽  
...  
Keyword(s):  
Image Analysis ◽  
Object Detection ◽  
Real Time ◽  
Medical Image ◽  
Medical Image Analysis ◽  
Semantic Segmentation ◽  
Autonomous Driving ◽  
Joint Detection ◽  
Multi Scale ◽  
Segmentation Task

Real-time semantic segmentation has become crucial in many applications such as medical image analysis and autonomous driving. In this paper, we introduce a single semantic segmentation network, called DNS, for joint object detection and segmentation task. We take advantage of multi-scale deconvolution mechanism to perform real time computations. To this goal, down-scale and up-scale streams are utilized to combine the multi-scale features for the final detection and segmentation task. By using the proposed DNS, not only the tradeoff between accuracy and cost but also the balance of detection and segmentation performance are settled. Experimental results for PASCAL VOC datasets show competitive performance for joint object detection and segmentation task.

scholarly journals REAL-TIME SEMANTIC SLAM WITH DCNN-BASED FEATURE POINT DETECTION, MATCHING AND DENSE POINT CLOUD AGGREGATION

2021 ◽  
Vol XLIII-B2-2021 ◽  
pp. 399-404
Author(s):  
B. Vishnyakov ◽  
I. Sgibnev ◽  
V. Sheverdin ◽  
A. Sorokin ◽  
P. Masalov ◽  
...  
Keyword(s):  
Neural Networks ◽  
Real Time ◽  
Semantic Segmentation ◽  
Autonomous Driving ◽  
Scene Reconstruction ◽  
Deep Convolutional Neural Networks ◽  
Dense Point ◽  
Robotic Vehicle ◽  
Semantic Scene ◽  
Point Detection

Abstract. In this paper we present the semantic SLAM method based on a bundle of deep convolutional neural networks. It provides real-time dense semantic scene reconstruction for the autonomous driving system of an off-road robotic vehicle. Most state-of-the-art neural networks require large computing resources that go beyond the capabilities of many robotic platforms. We propose an architecture for 3D semantic scene reconstruction on top of the recent progress in computer vision by integrating SuperPoint, SuperGlue, Bi3D, DeepLabV3+, RTM3D and additional module with pre-processing, inference and postprocessing operations performed on GPU. We also updated our simulated dataset for semantic segmentation and added disparity images.

scholarly journals Real-Time HD Map Change Detection for Crowdsourcing Update Based on Mid-to-High-End Sensors

Sensors ◽  
2021 ◽  
Vol 21 (7) ◽  
pp. 2477
Author(s):  
Pan Zhang ◽  
Mingming Zhang ◽  
Jingnan Liu
Keyword(s):  
Change Detection ◽  
Real Time ◽  
Point Cloud ◽  
Measurement Unit ◽  
Point Cloud Data ◽  
Integrated Navigation ◽  
High Definition ◽  
Cloud Data ◽  
Computing Platform ◽  
Semantic Point

Continuous maintenance and real-time update of high-definition (HD) maps is a big challenge. With the development of autonomous driving, more and more vehicles are equipped with a variety of advanced sensors and a powerful computing platform. Based on mid-to-high-end sensors including an industry camera, a high-end Global Navigation Satellite System (GNSS)/Inertial Measurement Unit (IMU), and an onboard computing platform, a real-time HD map change detection method for crowdsourcing update is proposed in this paper. First, a mature commercial integrated navigation product is directly used to achieve a self-positioning accuracy of 20 cm on average. Second, an improved network based on BiSeNet is utilized for real-time semantic segmentation. It achieves the result of 83.9% IOU (Intersection over Union) on Nvidia Pegasus at 31 FPS. Third, a visual Simultaneous Localization and Mapping (SLAM) associated with pixel type information is performed to obtain the semantic point cloud data of features such as lane dividers, road markings, and other static objects. Finally, the semantic point cloud data is vectorized after denoising and clustering, and the results are matched with a pre-constructed HD map to confirm map elements that have not changed and generate new elements when appearing. The experiment conducted in Beijing shows that the method proposed is effective for crowdsourcing update of HD maps.

scholarly journals A Vision/Inertia Integrated Positioning Method Using Position and Orientation Matching

2017 ◽  
Vol 2017 ◽  
pp. 1-11 ◽  
Author(s):  
Xiaoyue Zhang ◽  
Liang Huo
Keyword(s):  
Real Time ◽  
Inertial Measurement Unit ◽  
Measurement Unit ◽  
Integrated Navigation ◽  
Test Results ◽  
Position Information ◽  
Landmark Navigation ◽  
Positioning Method ◽  
Position And Orientation ◽  
Navigation Errors

A vision/inertia integrated positioning method using position and orientation matching which can be adopted on intelligent vehicle such as automated guided vehicle (AGV) and mobile robot is proposed in this work. The method is introduced firstly. Landmarks are placed into the navigation field and camera and inertial measurement unit (IMU) are installed on the vehicle. Vision processor calculates the azimuth and position information from the pictures which include artificial landmarks with the known direction and position. Inertial navigation system (INS) calculates the azimuth and position of vehicle in real time and the calculated pixel position of landmark can be computed from the INS output position. Then the needed mathematical models are established and integrated navigation is implemented by Kalman filter with the observation of azimuth and the calculated pixel position of landmark. Navigation errors and IMU errors are estimated and compensated in real time so that high precision navigation results can be got. Finally, simulation and test are performed, respectively. Both simulation and test results prove that this vision/inertia integrated positioning method using position and orientation matching has feasibility and it can achieve centimeter-level autonomic continuous navigation.

Sign in / Sign up

Export Citation Format

Share Document