scholarly journals Combined Edge- and Stixel-based Object Detection in 3D Point Cloud

Sensors ◽  
2019 ◽  
Vol 19 (20) ◽  
pp. 4423 ◽  
Author(s):  
Hu ◽  
Yang ◽  
Li
Keyword(s):  
Autonomous Vehicles ◽  
Point Cloud ◽  
Three Dimensional ◽  
Point Clouds ◽  
Computational Time ◽  
Measurement Unit ◽  
3D Point Cloud ◽  
Safe Driving ◽  
Driving Assistance ◽  
Feasible Path

Environment perception is critical for feasible path planning and safe driving for autonomous vehicles. Perception devices, such as camera, LiDAR (Light Detection and Ranging), IMU(Inertial Measurement Unit), etc., only provide raw sensing data with no identification of vital objects, which is insufficient for autonomous vehicles to perform safe and efficient self-driving operations. This study proposes an improved edge-oriented segmentation-based method to detect the objects from the sensed three-dimensional (3D) point cloud. The improved edge-oriented segmentation-based method consists of three main steps: First, the bounding areas of objects are identified by edge detection and stixel estimation in corresponding two-dimensional (2D) images taken by a stereo camera. Second, 3D sparse point clouds of objects are reconstructed in bounding areas. Finally, the dense point clouds of objects are segmented by matching the 3D sparse point clouds of objects with the whole scene point cloud. After comparison with the existing methods of segmentation, the experimental results demonstrate that the proposed edge-oriented segmentation method improves the precision of 3D point cloud segmentation, and that the objects can be segmented accurately. Meanwhile, the visualization of output data in advanced driving assistance systems (ADAS) can be greatly facilitated due to the decrease in computational time and the decrease in the number of points in the object’s point cloud.

scholarly journals Nondestructive Determination of Nitrogen, Phosphorus and Potassium Contents in Greenhouse Tomato Plants Based on Multispectral Three-Dimensional Imaging

Sensors ◽  
2019 ◽  
Vol 19 (23) ◽  
pp. 5295 ◽  
Author(s):  
Guoxiang Sun ◽  
Yongqian Ding ◽  
Xiaochan Wang ◽  
Wei Lu ◽  
Ye Sun ◽  
...  
Keyword(s):  
Fourier Transform ◽  
Point Cloud ◽  
Prediction Models ◽  
Three Dimensional ◽  
Point Clouds ◽  
Support Vector ◽  
3D Point Cloud ◽  
Tomato Plants ◽  
Nondestructive Determination ◽  
Cloud Models

Measurement of plant nitrogen (N), phosphorus (P), and potassium (K) levels are important for determining precise fertilization management approaches for crops cultivated in greenhouses. To accurately, rapidly, stably, and nondestructively measure the NPK levels in tomato plants, a nondestructive determination method based on multispectral three-dimensional (3D) imaging was proposed. Multiview RGB-D images and multispectral images were synchronously collected, and the plant multispectral reflectance was registered to the depth coordinates according to Fourier transform principles. Based on the Kinect sensor pose estimation and self-calibration, the unified transformation of the multiview point cloud coordinate system was realized. Finally, the iterative closest point (ICP) algorithm was used for the precise registration of multiview point clouds and the reconstruction of plant multispectral 3D point cloud models. Using the normalized grayscale similarity coefficient, the degree of spectral overlap, and the Hausdorff distance set, the accuracy of the reconstructed multispectral 3D point clouds was quantitatively evaluated, the average value was 0.9116, 0.9343 and 0.41 cm, respectively. The results indicated that the multispectral reflectance could be registered to the Kinect depth coordinates accurately based on the Fourier transform principles, the reconstruction accuracy of the multispectral 3D point cloud model met the model reconstruction needs of tomato plants. Using back-propagation artificial neural network (BPANN), support vector machine regression (SVMR), and gaussian process regression (GPR) methods, determination models for the NPK contents in tomato plants based on the reflectance characteristics of plant multispectral 3D point cloud models were separately constructed. The relative error (RE) of the N content by BPANN, SVMR and GPR prediction models were 2.27%, 7.46% and 4.03%, respectively. The RE of the P content by BPANN, SVMR and GPR prediction models were 3.32%, 8.92% and 8.41%, respectively. The RE of the K content by BPANN, SVMR and GPR prediction models were 3.27%, 5.73% and 3.32%, respectively. These models provided highly efficient and accurate measurements of the NPK contents in tomato plants. The NPK contents determination performance of these models were more stable than those of single-view models.

scholarly journals Sloped Terrain Segmentation for Autonomous Drive Using Sparse 3D Point Cloud

2014 ◽  
Vol 2014 ◽  
pp. 1-9 ◽  
Author(s):  
Seoungjae Cho ◽  
Jonghyun Kim ◽  
Warda Ikram ◽  
Kyungeun Cho ◽  
Young-Sik Jeong ◽  
...  
Keyword(s):  
Real Time ◽  
Point Cloud ◽  
Data Exchange ◽  
Three Dimensional ◽  
Point Clouds ◽  
Two Dimensions ◽  
3D Point Cloud ◽  
3D Point Clouds ◽  
Ubiquitous Environment ◽  
Ground Segmentation

A ubiquitous environment for road travel that uses wireless networks requires the minimization of data exchange between vehicles. An algorithm that can segment the ground in real time is necessary to obtain location data between vehicles simultaneously executing autonomous drive. This paper proposes a framework for segmenting the ground in real time using a sparse three-dimensional (3D) point cloud acquired from undulating terrain. A sparse 3D point cloud can be acquired by scanning the geography using light detection and ranging (LiDAR) sensors. For efficient ground segmentation, 3D point clouds are quantized in units of volume pixels (voxels) and overlapping data is eliminated. We reduce nonoverlapping voxels to two dimensions by implementing a lowermost heightmap. The ground area is determined on the basis of the number of voxels in each voxel group. We execute ground segmentation in real time by proposing an approach to minimize the comparison between neighboring voxels. Furthermore, we experimentally verify that ground segmentation can be executed at about 19.31 ms per frame.

scholarly journals HoPE: Horizontal Plane Extractor for Cluttered 3D Scenes

Sensors ◽  
2018 ◽  
Vol 18 (10) ◽  
pp. 3214 ◽  
Author(s):  
Zhipeng Dong ◽  
Yi Gao ◽  
Jinfeng Zhang ◽  
Yunhui Yan ◽  
Xin Wang ◽  
...  
Keyword(s):  
Horizontal Plane ◽  
Point Cloud ◽  
Nearest Neighbor ◽  
Three Dimensional ◽  
Principal Component ◽  
Region Growing ◽  
Point Clouds ◽  
Measurement Unit ◽  
Coordinate Frame ◽  
3D Point Clouds

Extracting horizontal planes in heavily cluttered three-dimensional (3D) scenes is an essential procedure for many robotic applications. Aiming at the limitations of general plane segmentation methods on this subject, we present HoPE, a Horizontal Plane Extractor that is able to extract multiple horizontal planes in cluttered scenes with both organized and unorganized 3D point clouds. It transforms the source point cloud in the first stage to the reference coordinate frame using the sensor orientation acquired either by pre-calibration or an inertial measurement unit, thereby leveraging the inner structure of the transformed point cloud to ease the subsequent processes that use two concise thresholds for producing the results. A revised region growing algorithm named Z clustering and a principal component analysis (PCA)-based approach are presented for point clustering and refinement, respectively. Furthermore, we provide a nearest neighbor plane matching (NNPM) strategy to preserve the identities of extracted planes across successive sequences. Qualitative and quantitative evaluations of both real and synthetic scenes demonstrate that our approach outperforms several state-of-the-art methods under challenging circumstances, in terms of robustness to clutter, accuracy, and efficiency. We make our algorithm an off-the-shelf toolbox which is publicly available.

scholarly journals DeepLabV3-Refiner-Based Semantic Segmentation Model for Dense 3D Point Clouds

2021 ◽  
Vol 13 (8) ◽  
pp. 1565
Author(s):  
Jeonghoon Kwak ◽  
Yunsick Sung
Keyword(s):  
Virtual Environment ◽  
Point Cloud ◽  
Three Dimensional ◽  
Fold Increase ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
3D Point Cloud ◽  
3D Point Clouds ◽  
Segmented Image ◽  
Human Object

Three-dimensional virtual environments can be configured as test environments of autonomous things, and remote sensing by 3D point clouds collected by light detection and range (LiDAR) can be used to detect virtual human objects by segmenting collected 3D point clouds in a virtual environment. The use of a traditional encoder-decoder model, such as DeepLabV3, improves the quality of the low-density 3D point clouds of human objects, where the quality is determined by the measurement gap of the LiDAR lasers. However, whenever a human object with a surrounding environment in a 3D point cloud is used by the traditional encoder-decoder model, it is difficult to increase the density fitting of the human object. This paper proposes a DeepLabV3-Refiner model, which is a model that refines the fit of human objects using human objects whose density has been increased through DeepLabV3. An RGB image that has a segmented human object is defined as a dense segmented image. DeepLabV3 is used to make predictions of dense segmented images and 3D point clouds for human objects in 3D point clouds. In the Refiner model, the results of DeepLabV3 are refined to fit human objects, and a dense segmented image fit to human objects is predicted. The dense 3D point cloud is calculated using the dense segmented image provided by the DeepLabV3-Refiner model. The 3D point clouds that were analyzed by the DeepLabV3-Refiner model had a 4-fold increase in density, which was verified experimentally. The proposed method had a 0.6% increase in density accuracy compared to that of DeepLabV3, and a 2.8-fold increase in the density corresponding to the human object. The proposed method was able to provide a 3D point cloud that increased the density to fit the human object. The proposed method can be used to provide an accurate 3D virtual environment by using the improved 3D point clouds.

scholarly journals Cost Effective Mobile Mapping System for Color Point Cloud Reconstruction

Sensors ◽  
2020 ◽  
Vol 20 (22) ◽  
pp. 6536
Author(s):  
Cheng-Wei Peng ◽  
Chen-Chien Hsu ◽  
Wei-Yen Wang
Keyword(s):  
Autonomous Vehicles ◽  
Point Cloud ◽  
Cost Effective ◽  
Point Clouds ◽  
Autonomous Driving ◽  
Fine Tuning ◽  
Measurement Unit ◽  
Practical Implementation ◽  
High Definition ◽  
Mobile Mapping

Survey-grade Lidar brands have commercialized Lidar-based mobile mapping systems (MMSs) for several years now. With this high-end equipment, the high-level accuracy quality of point clouds can be ensured, but unfortunately, their high cost has prevented practical implementation in autonomous driving from being affordable. As an attempt to solve this problem, we present a cost-effective MMS to generate an accurate 3D color point cloud for autonomous vehicles. Among the major processes for color point cloud reconstruction, we first synchronize the timestamps of each sensor. The calibration process between camera and Lidar is developed to obtain the translation and rotation matrices, based on which color attributes can be composed into the corresponding Lidar points. We also employ control points to adjust the point cloud for fine tuning the absolute position. To overcome the limitation of Global Navigation Satellite System/Inertial Measurement Unit (GNSS/IMU) positioning system, we utilize Normal Distribution Transform (NDT) localization to refine the trajectory to solve the multi-scan dispersion issue. Experimental results show that the color point cloud reconstructed by the proposed MMS has a position error in centimeter-level accuracy, meeting the requirement of high definition (HD) maps for autonomous driving usage.

scholarly journals A DIGITAL ARCHIVE OF BOROBUDUR BASED ON 3D POINT CLOUDS

2021 ◽  
Vol XLIII-B2-2021 ◽  
pp. 577-582
Author(s):  
M. Kawato ◽  
L. Li ◽  
K. Hasegawa ◽  
M. Adachi ◽  
H. Yamaguchi ◽  
...  
Keyword(s):  
Cultural Heritage ◽  
Point Cloud ◽  
Three Dimensional ◽  
Point Clouds ◽  
World Heritage Site ◽  
Digital Archive ◽  
First Person ◽  
3D Point Cloud ◽  
3D Point Clouds ◽  
The Temple

Abstract. Three-dimensional point clouds are becoming popular representations for digital archives of cultural heritage sites. The Borobudur Temple, located in Central Java, Indonesia, was built in the 8th century. Borobudur is considered one of the greatest Buddhist monuments in the world and was listed as a UNESCO World Heritage site. We are developing a virtual reality system as a digital archive of the Borobudur Temple. This research is a collaboration between Ritsumeikan University, Japan, the Indonesian Institute of Sciences (LIPI), and the Borobudur Conservation Office, Indonesia. In our VR system, the following three data sources are integrated to form a 3D point cloud: (1) a 3D point cloud of the overall shape of the temple acquired by photogrammetry using a camera carried by a UAV, (2) a 3D point cloud obtained from precise photogrammetric measurements of selected parts of the temple building, and (3) 3D data of the hidden relief panels recovered from the archived 2D monocular photos using deep learning. Our VR system supports both the first-person view and the bird’s eye view. The first-person view allows immersive observation and appreciation of the cultural heritage. The bird’s eye view is useful for understanding the whole picture. A user can easily switch between the two views by using a user-friendly VR user interface constructed by a 3D game engine.

Extraction of Road Lane Markings from Mobile LiDAR Data

2021 ◽  
pp. 036119812098194
Author(s):  
Mustafa Zeybek
Keyword(s):  
Traffic Management ◽  
Point Cloud ◽  
Three Dimensional ◽  
Point Clouds ◽  
Lidar Data ◽  
Safe Driving ◽  
Cloud Data ◽  
Detection Techniques ◽  
On The Road ◽  
The Right

This study presents a method for automatic extraction of road lane markings from mobile light detection and ranging (LiDAR) data. Road lanes and traffic signs on the road surface provide safe driving for drivers and aid traffic flow movement along the highway and street. Mobile LiDAR systems acquire massive datasets very quickly in a short time. To simplify the data structure and feature extraction, it is essential for traffic management personnel to apply the right methods. Road lanes must be visible and are a major factor in road safety for drivers. In this study, a methodology is devised and implemented for the extraction of features such as dashed lines, continuous lanes, and direction arrows on the pavement from point clouds. Point cloud data was collected from the Riegl VMX-450 mobile LiDAR system. The alpha shape algorithm is implemented on a point cloud and compared with the widespread use of edge detection techniques applied for intensity-based raster images. The proposed methodology directly extracts three-dimensional and two-dimensional road features to control the quality of road markings and spatial positions with the obtained marking boundaries. State-of-the-art results are obtained and compared with manually digitized reference markings. The standard deviations were evaluated and acquired for intensity image-based and direct point cloud-based extractions, at 1.2 cm and 1.7 cm, respectively.

scholarly journals Matching Point Clouds with STL Models by Using the Principle Component Analysis and a Decomposition into Geometric Primitives

2021 ◽  
Vol 11 (5) ◽  
pp. 2268
Author(s):  
Erika Straková ◽  
Dalibor Lukáš ◽  
Zdenko Bobovský ◽  
Tomáš Kot ◽  
Milan Mihola ◽  
...  
Keyword(s):  
Point Cloud ◽  
Principal Component ◽  
Point Clouds ◽  
Component Analysis ◽  
Language Models ◽  
Computational Time ◽  
Dominant Eigenvalue ◽  
Eigenvalues And Eigenvectors ◽  
Correct Model ◽  
Geometric Primitives

While repairing industrial machines or vehicles, recognition of components is a critical and time-consuming task for a human. In this paper, we propose to automatize this task. We start with a Principal Component Analysis (PCA), which fits the scanned point cloud with an ellipsoid by computing the eigenvalues and eigenvectors of a 3-by-3 covariant matrix. In case there is a dominant eigenvalue, the point cloud is decomposed into two clusters to which the PCA is applied recursively. In case the matching is not unique, we continue to distinguish among several candidates. We decompose the point cloud into planar and cylindrical primitives and assign mutual features such as distance or angle to them. Finally, we refine the matching by comparing the matrices of mutual features of the primitives. This is a more computationally demanding but very robust method. We demonstrate the efficiency and robustness of the proposed methodology on a collection of 29 real scans and a database of 389 STL (Standard Triangle Language) models. As many as 27 scans are uniquely matched to their counterparts from the database, while in the remaining two cases, there is only one additional candidate besides the correct model. The overall computational time is about 10 min in MATLAB.

scholarly journals Fitting Terrestrial Laser Scanner Point Clouds with T-Splines: Local Refinement Strategy for Rigid Body Motion

2021 ◽  
Vol 13 (13) ◽  
pp. 2494
Author(s):  
Gaël Kermarrec ◽  
Niklas Schild ◽  
Jan Hartmann
Keyword(s):  
Point Cloud ◽  
Goodness Of Fit ◽  
Laser Scanner ◽  
Point Clouds ◽  
Statistical Testing ◽  
Computational Time ◽  
Terrestrial Laser Scanner ◽  
Reference Surface ◽  
Local Refinement ◽  
Surface Approximation

T-splines have recently been introduced to represent objects of arbitrary shapes using a smaller number of control points than the conventional non-uniform rational B-splines (NURBS) or B-spline representatizons in computer-aided design, computer graphics and reverse engineering. They are flexible in representing complex surface shapes and economic in terms of parameters as they enable local refinement. This property is a great advantage when dense, scattered and noisy point clouds are approximated using least squares fitting, such as those from a terrestrial laser scanner (TLS). Unfortunately, when it comes to assessing the goodness of fit of the surface approximation with a real dataset, only a noisy point cloud can be approximated: (i) a low root mean squared error (RMSE) can be linked with an overfitting, i.e., a fitting of the noise, and should be correspondingly avoided, and (ii) a high RMSE is synonymous with a lack of details. To address the challenge of judging the approximation, the reference surface should be entirely known: this can be solved by printing a mathematically defined T-splines reference surface in three dimensions (3D) and modeling the artefacts induced by the 3D printing. Once scanned under different configurations, it is possible to assess the goodness of fit of the approximation for a noisy and potentially gappy point cloud and compare it with the traditional but less flexible NURBS. The advantages of T-splines local refinement open the door for further applications within a geodetic context such as rigorous statistical testing of deformation. Two different scans from a slightly deformed object were approximated; we found that more than 40% of the computational time could be saved without affecting the goodness of fit of the surface approximation by using the same mesh for the two epochs.

scholarly journals Multi-Dimensional Underwater Point Cloud Detection Based on Deep Learning

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 884
Author(s):  
Chia-Ming Tsai ◽  
Yi-Horng Lai ◽  
Yung-Da Sun ◽  
Yu-Jen Chung ◽  
Jau-Woei Perng
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Three Dimensional ◽  
Point Clouds ◽  
Training Data ◽  
Network Architectures ◽  
Point Cloud Data ◽  
Data Types ◽  
Raw Data ◽  
Cloud Data

Numerous sensors can obtain images or point cloud data on land, however, the rapid attenuation of electromagnetic signals and the lack of light in water have been observed to restrict sensing functions. This study expands the utilization of two- and three-dimensional detection technologies in underwater applications to detect abandoned tires. A three-dimensional acoustic sensor, the BV5000, is used in this study to collect underwater point cloud data. Some pre-processing steps are proposed to remove noise and the seabed from raw data. Point clouds are then processed to obtain two data types: a 2D image and a 3D point cloud. Deep learning methods with different dimensions are used to train the models. In the two-dimensional method, the point cloud is transferred into a bird’s eye view image. The Faster R-CNN and YOLOv3 network architectures are used to detect tires. Meanwhile, in the three-dimensional method, the point cloud associated with a tire is cut out from the raw data and is used as training data. The PointNet and PointConv network architectures are then used for tire classification. The results show that both approaches provide good accuracy.

Sign in / Sign up

Export Citation Format

Share Document