scholarly journals Two-Layer-Graph Clustering for Real-Time 3D LiDAR Point Cloud Segmentation

2020 ◽  
Vol 10 (23) ◽  
pp. 8534
Author(s):  
Haozhe Yang ◽  
Zhiling Wang ◽  
Linglong Lin ◽  
Huawei Liang ◽  
Weixin Huang ◽  
...  
Keyword(s):  
Real Time ◽  
Autonomous Vehicles ◽  
Point Cloud ◽  
Processing Time ◽  
Autonomous Vehicle ◽  
Point Clouds ◽  
Complex Objects ◽  
Time Performance ◽  
Sensor Position ◽  
Perception System

The perception system has become a topic of great importance for autonomous vehicles, as high accuracy and real-time performance can ensure safety in complex urban scenarios. Clustering is a fundamental step for parsing point cloud due to the extensive input data (over 100,000 points) of a wide variety of complex objects. It is still challenging to achieve high precision real-time performance with limited vehicle-mounted computing resources, which need to balance the accuracy and processing time. We propose a method based on a Two-Layer-Graph (TLG) structure, which can be applied in a real autonomous vehicle under urban scenarios. TLG can describe the point clouds hierarchically, we use a range graph to represent point clouds and a set graph for point cloud sets, which reduce both processing time and memory consumption. In the range graph, Euclidean distance and the angle of the sensor position with two adjacent vectors (calculated from continuing points to different direction) are used as the segmentation standard, which use the local concave features to distinguish different objects close to each other. In the set graph, we use the start and end position to express the whole set of continuous points concisely, and an improved Breadth-First-Search (BFS) algorithm is designed to update categories of point cloud sets between different channels. This method is evaluated on real vehicles and major datasets. The results show that TLG succeeds in providing a real-time performance (less than 20 ms per frame), and a high segmentation accuracy rate (93.64%) for traffic objects in the road of urban scenarios.

scholarly journals 3D Fast Object Detection Based on Discriminant Images and Dynamic Distance Threshold Clustering

Sensors ◽  
2020 ◽  
Vol 20 (24) ◽  
pp. 7221
Author(s):  
Baifan Chen ◽  
Hong Chen ◽  
Dian Yuan ◽  
Lingli Yu
Keyword(s):  
Object Detection ◽  
Real Time ◽  
Autonomous Vehicles ◽  
Point Cloud ◽  
Point Cloud Data ◽  
Long Distance ◽  
Safe Driving ◽  
Distance Threshold ◽  
Cloud Data ◽  
Perception System

The object detection algorithm based on vehicle-mounted lidar is a key component of the perception system on autonomous vehicles. It can provide high-precision and highly robust obstacle information for the safe driving of autonomous vehicles. However, most algorithms are often based on a large amount of point cloud data, which makes real-time detection difficult. To solve this problem, this paper proposes a 3D fast object detection method based on three main steps: First, the ground segmentation by discriminant image (GSDI) method is used to convert point cloud data into discriminant images for ground points segmentation, which avoids the direct computing of the point cloud data and improves the efficiency of ground points segmentation. Second, the image detector is used to generate the region of interest of the three-dimensional object, which effectively narrows the search range. Finally, the dynamic distance threshold clustering (DDTC) method is designed for different density of the point cloud data, which improves the detection effect of long-distance objects and avoids the over-segmentation phenomenon generated by the traditional algorithm. Experiments have showed that this algorithm can meet the real-time requirements of autonomous driving while maintaining high accuracy.

Real-Time 3D object detection using improved convolutional neural network based on image-driven point cloud

2021 ◽  
Vol 14 ◽  
Author(s):  
Zhiyong Gao ◽  
Jianhong Xiang
Keyword(s):  
Neural Network ◽  
Object Detection ◽  
Convolutional Neural Network ◽  
Real Time ◽  
Point Cloud ◽  
Point Clouds ◽  
3D Point Cloud ◽  
3D Object ◽  
3D Object Detection ◽  
Instance Segmentation

Background: While detecting the object directly from the 3D point cloud, the natural 3D patterns and invariance of 3D data are often obscure. Objective: In this work, we aimed at studying the 3D object detection from discrete, disordered and sparse 3D point clouds. Methods: The CNN is composed of the frustum sequence module, 3D instance segmentation module S-NET, 3D point cloud transformation module T-NET, and 3D boundary box estimation module E-NET. The search space of the object is determined by the frustum sequence module. The instance segmentation of the point cloud is performed by the 3D instance segmentation module. The 3D coordinates of the object are confirmed by the transformation module and the 3D bounding box estimation module. Results: Evaluated on KITTI benchmark dataset, our method outperforms the state of the art by remarkable margins while having real-time capability. Conclusion: We achieve real-time 3D object detection by proposing an improved convolutional neural network (CNN) based on image-driven point clouds.

scholarly journals Performance Analysis of NDT-based Graph SLAM for Autonomous Vehicle in Diverse Typical Driving Scenarios of Hong Kong

Sensors ◽  
2018 ◽  
Vol 18 (11) ◽  
pp. 3928 ◽  
Author(s):  
Weisong Wen ◽  
Li-Ta Hsu ◽  
Guohao Zhang
Keyword(s):  
New York ◽  
Hong Kong ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Tall Buildings ◽  
Point Clouds ◽  
Reliability Estimation ◽  
Traffic Condition ◽  
Localization And Mapping ◽  
Graph Slam

Robust and lane-level positioning is essential for autonomous vehicles. As an irreplaceable sensor, Light detection and ranging (LiDAR) can provide continuous and high-frequency pose estimation by means of mapping, on condition that enough environment features are available. The error of mapping can accumulate over time. Therefore, LiDAR is usually integrated with other sensors. In diverse urban scenarios, the environment feature availability relies heavily on the traffic (moving and static objects) and the degree of urbanization. Common LiDAR-based simultaneous localization and mapping (SLAM) demonstrations tend to be studied in light traffic and less urbanized area. However, its performance can be severely challenged in deep urbanized cities, such as Hong Kong, Tokyo, and New York with dense traffic and tall buildings. This paper proposes to analyze the performance of standalone NDT-based graph SLAM and its reliability estimation in diverse urban scenarios to further evaluate the relationship between the performance of LiDAR-based SLAM and scenario conditions. The normal distribution transform (NDT) is employed to calculate the transformation between frames of point clouds. Then, the LiDAR odometry is performed based on the calculated continuous transformation. The state-of-the-art graph-based optimization is used to integrate the LiDAR odometry measurements to implement optimization. The 3D building models are generated and the definition of the degree of urbanization based on Skyplot is proposed. Experiments are implemented in different scenarios with different degrees of urbanization and traffic conditions. The results show that the performance of the LiDAR-based SLAM using NDT is strongly related to the traffic condition and degree of urbanization. The best performance is achieved in the sparse area with normal traffic and the worse performance is obtained in dense urban area with 3D positioning error (summation of horizontal and vertical) gradients of 0.024 m/s and 0.189 m/s, respectively. The analyzed results can be a comprehensive benchmark for evaluating the performance of standalone NDT-based graph SLAM in diverse scenarios which is significant for multi-sensor fusion of autonomous vehicle.

scholarly journals Principal Component Analysis For ICP Pose Estimation Of Space Structures

2021 ◽  
Author(s):  
Lun H. Mark
Keyword(s):  
Pose Estimation ◽  
Point Cloud ◽  
Principal Component ◽  
Point Clouds ◽  
Optimal Combination ◽  
Estimation Accuracy ◽  
Space Structures ◽  
Complex Objects ◽  
Error Norm ◽  
Simulation Results

This thesis investigates how geometry of complex objects is related to LIDAR scanning with the Iterative Closest Point (ICP) pose estimation and provides statistical means to assess the pose accuracy. LIDAR scanners have become essential parts of space vision systems for autonomous docking and rendezvous. Principal Componenet Analysis based geometric constraint indices have been found to be strongly related to the pose error norm and the error of each individual degree of freedom. This leads to the development of several strategies for identifying the best view of an object and the optimal combination of localized scanned areas of the object's surface to achieve accurate pose estimation. Also investigated is the possible relation between the ICP pose estimation accuracy and the districution or allocation of the point cloud. The simulation results were validated using point clouds generated by scanning models of Quicksat and a cuboctahedron using Neptec's TriDAR scanner.

scholarly journals Principal Component Analysis For ICP Pose Estimation Of Space Structures

2021 ◽  
Author(s):  
Lun H. Mark
Keyword(s):  
Pose Estimation ◽  
Point Cloud ◽  
Principal Component ◽  
Point Clouds ◽  
Optimal Combination ◽  
Estimation Accuracy ◽  
Space Structures ◽  
Complex Objects ◽  
Error Norm ◽  
Simulation Results

This thesis investigates how geometry of complex objects is related to LIDAR scanning with the Iterative Closest Point (ICP) pose estimation and provides statistical means to assess the pose accuracy. LIDAR scanners have become essential parts of space vision systems for autonomous docking and rendezvous. Principal Componenet Analysis based geometric constraint indices have been found to be strongly related to the pose error norm and the error of each individual degree of freedom. This leads to the development of several strategies for identifying the best view of an object and the optimal combination of localized scanned areas of the object's surface to achieve accurate pose estimation. Also investigated is the possible relation between the ICP pose estimation accuracy and the districution or allocation of the point cloud. The simulation results were validated using point clouds generated by scanning models of Quicksat and a cuboctahedron using Neptec's TriDAR scanner.

scholarly journals Computer vision based obstacle detection and target tracking for autonomous vehicles

2021 ◽  
Vol 336 ◽  
pp. 07004
Author(s):  
Ruoyu Fang ◽  
Cheng Cai
Keyword(s):  
Neural Network ◽  
Computer Vision ◽  
Deep Learning ◽  
Target Tracking ◽  
Real Time ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Obstacle Detection ◽  
Pid Algorithm ◽  
Deep Learning Neural Network

Obstacle detection and target tracking are two major issues for intelligent autonomous vehicles. This paper proposes a new scheme to achieve target tracking and real-time obstacle detection of obstacles based on computer vision. ResNet-18 deep learning neural network is utilized for obstacle detection and Yolo-v3 deep learning neural network is employed for real-time target tracking. These two trained models can be deployed on an autonomous vehicle equipped with an NVIDIA Jetson Nano motherboard. The autonomous vehicle moves to avoid obstacles and follow tracked targets by camera. Adjusting the steering and movement of the autonomous vehicle according to the PID algorithm during the movement, therefore, will help the proposed vehicle achieve stable and precise tracking.

scholarly journals Design and Implementation of Edge-Fog-Cloud System through HD Map Generation from LiDAR Data of Autonomous Vehicles

Electronics ◽  
2020 ◽  
Vol 9 (12) ◽  
pp. 2084
Author(s):  
Junwon Lee ◽  
Kieun Lee ◽  
Aelee Yoo ◽  
Changjoo Moon
Keyword(s):  
Real Time ◽  
Autonomous Vehicles ◽  
Fog Computing ◽  
Point Clouds ◽  
Lidar Data ◽  
High Definition ◽  
Localization And Mapping ◽  
Cloud Server ◽  
Map Generation ◽  
Self Driving Cars

Self-driving cars, autonomous vehicles (AVs), and connected cars combine the Internet of Things (IoT) and automobile technologies, thus contributing to the development of society. However, processing the big data generated by AVs is a challenge due to overloading issues. Additionally, near real-time/real-time IoT services play a significant role in vehicle safety. Therefore, the architecture of an IoT system that collects and processes data, and provides services for vehicle driving, is an important consideration. In this study, we propose a fog computing server model that generates a high-definition (HD) map using light detection and ranging (LiDAR) data generated from an AV. The driving vehicle edge node transmits the LiDAR point cloud information to the fog server through a wireless network. The fog server generates an HD map by applying the Normal Distribution Transform-Simultaneous Localization and Mapping(NDT-SLAM) algorithm to the point clouds transmitted from the multiple edge nodes. Subsequently, the coordinate information of the HD map generated in the sensor frame is converted to the coordinate information of the global frame and transmitted to the cloud server. Then, the cloud server creates an HD map by integrating the collected point clouds using coordinate information.

scholarly journals Real-Time Vehicle Detection Algorithm Based on Vision and Lidar Point Cloud Fusion

2019 ◽  
Vol 2019 ◽  
pp. 1-9 ◽  
Author(s):  
Hai Wang ◽  
Xinyu Lou ◽  
Yingfeng Cai ◽  
Yicheng Li ◽  
Long Chen
Keyword(s):  
Deep Learning ◽  
Real Time ◽  
Autonomous Vehicles ◽  
Point Cloud ◽  
Vehicle Detection ◽  
Detection Algorithm ◽  
Detection Methods ◽  
Depth Information ◽  
Detection Accuracy ◽  
Classification Rate

Vehicle detection is one of the most important environment perception tasks for autonomous vehicles. The traditional vision-based vehicle detection methods are not accurate enough especially for small and occluded targets, while the light detection and ranging- (lidar-) based methods are good in detecting obstacles but they are time-consuming and have a low classification rate for different target types. Focusing on these shortcomings to make the full use of the advantages of the depth information of lidar and the obstacle classification ability of vision, this work proposes a real-time vehicle detection algorithm which fuses vision and lidar point cloud information. Firstly, the obstacles are detected by the grid projection method using the lidar point cloud information. Then, the obstacles are mapped to the image to get several separated regions of interest (ROIs). After that, the ROIs are expanded based on the dynamic threshold and merged to generate the final ROI. Finally, a deep learning method named You Only Look Once (YOLO) is applied on the ROI to detect vehicles. The experimental results on the KITTI dataset demonstrate that the proposed algorithm has high detection accuracy and good real-time performance. Compared with the detection method based only on the YOLO deep learning, the mean average precision (mAP) is increased by 17%.

scholarly journals SAGE: A S plit- A rchitecture Methodolo g y for E fficient End-to-End Autonomous Vehicle Control

2021 ◽  
Vol 20 (5s) ◽  
pp. 1-22
Author(s):  
Arnav Malawade ◽  
Mohanad Odema ◽  
Sebastien Lajeunesse-degroot ◽  
Mohammad Abdullah Al Faruque
Keyword(s):  
High Resolution ◽  
Real Time ◽  
Autonomous Vehicles ◽  
Autonomous Vehicle ◽  
Industry Standard ◽  
Wide Range ◽  
Energy Consuming ◽  
Time Latency ◽  
4G Lte ◽  
Hardware Platforms

Autonomous vehicles (AV) are expected to revolutionize transportation and improve road safety significantly. However, these benefits do not come without cost; AVs require large Deep-Learning (DL) models and powerful hardware platforms to operate reliably in real-time, requiring between several hundred watts to one kilowatt of power. This power consumption can dramatically reduce vehicles’ driving range and affect emissions. To address this problem, we propose SAGE: a methodology for selectively offloading the key energy-consuming modules of DL architectures to the cloud to optimize edge, energy usage while meeting real-time latency constraints. Furthermore, we leverage Head Network Distillation (HND) to introduce efficient bottlenecks within the DL architecture in order to minimize the network overhead costs of offloading with almost no degradation in the model’s performance. We evaluate SAGE using an Nvidia Jetson TX2 and an industry-standard Nvidia Drive PX2 as the AV edge, devices and demonstrate that our offloading strategy is practical for a wide range of DL models and internet connection bandwidths on 3G, 4G LTE, and WiFi technologies. Compared to edge-only computation, SAGE reduces energy consumption by an average of 36.13% , 47.07% , and 55.66% for an AV with one low-resolution camera, one high-resolution camera, and three high-resolution cameras, respectively. SAGE also reduces upload data size by up to 98.40% compared to direct camera offloading.

scholarly journals AUTOMATIC INTEGRATION OF LASER SCANNING AND PHOTOGRAMMETRIC POINT CLOUDS: FROM ACQUISITION TO CO-REGISTRATION

2021 ◽  
Vol XLIII-B1-2021 ◽  
pp. 85-92
Author(s):  
T. Partovi ◽  
M. Dähne ◽  
M. Maboudi ◽  
D. Krueger ◽  
M. Gerke
Keyword(s):  
Point Cloud ◽  
Laser Scanning ◽  
High Capacity ◽  
Point Clouds ◽  
Complex Objects ◽  
Initial Image ◽  
Joint Project ◽  
3D Point Clouds ◽  
Image Capturing ◽  
Data Capturing

Abstract. Laser scanning systems have been developed to capture very high-resolution 3D point clouds and consequently acquire the object geometry. This object measuring technique has a high capacity for being utilized in a wide variety of applications such as indoor and outdoor modelling. The Terrestrial Laser Scanning (TLS) is used as an important data capturing measurement system to provide high quality point cloud from industrial or built-up environments. However, the static nature of the TLS and complexity of the industrial sites necessitate employing a complementary data capturing system e.g. cameras to fill the gaps in the TLS point cloud caused by occlusions which is very common in complex industrial areas. Moreover, employing images provide better radiometric and edge information. This motivated a joint project to develop a system for automatic and robust co-registration of TLS data and images directly, especially for complex objects. In this paper, the proposed methods for various components of this project including gap detection from point cloud, calculation of initial image capturing configuration, user interface and support system for the image capturing procedures, and co-registration between TLS point cloud and photogrammetric point cloud are presented. The primarily results on a complex industrial environment are promising.

Sign in / Sign up

Export Citation Format

Share Document