scholarly journals Benchmarking Deep Learning Models on Point Cloud Segmentation

2021 ◽  
Author(s):  
Thanasis Zoumpekas ◽  
Guillem Molina ◽  
Maria Salamó ◽  
Anna Puig
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Performance Metrics ◽  
Point Clouds ◽  
Learning Models ◽  
Cloud Data ◽  
Point Cloud Segmentation ◽  
Segmentation Analysis ◽  
Intelligent Analysis ◽  
Segmentation Task

Point clouds are currently used for a variety of applications, such as detection tasks in medical and geological domains. Intelligent analysis of point clouds is considered a highly computationally demanding and challenging task, especially the segmentation task among the points. Although numerous deep learning models have recently been proposed to segment point cloud data, there is no clear instruction of which exactly neural network to utilize and then incorporate into a system dealing with point cloud segmentation analysis. Besides, the majority of the developed models emphasize more on the accuracy rather than the efficiency, in order to achieve great results. Consequently, the training, validation and testing phases of the models require a great number of processing hours and a huge amount of memory. These high computational requirements are commonly difficult to deal with for many users. In this article, we analyse five state-of-the-art deep learning models for part segmentation task and give meaningful insights into the utilization of each one. We advance guidelines based on different properties, considering both learning-related metrics, such as accuracy, and system-related metrics, such as run time and memory footprint. We further propose and analyse generalized performance metrics, which facilitate the model evaluation phase in segmentation tasks allowing users to select the most appropriate approach for their context in terms of accuracy and efficiency.

scholarly journals Label3DMaize: toolkit for 3D point cloud data annotation of maize shoots

GigaScience ◽  
2021 ◽  
Vol 10 (5) ◽  
Author(s):  
Teng Miao ◽  
Weiliang Wen ◽  
Yinglun Li ◽  
Sheng Wu ◽  
Chao Zhu ◽  
...  
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Training Dataset ◽  
Growth Stages ◽  
3D Point Cloud ◽  
Plant Structure ◽  
Data Annotation ◽  
Cloud Data ◽  
Point Cloud Segmentation ◽  
Different Growth Stages

Abstract Background The 3D point cloud is the most direct and effective data form for studying plant structure and morphology. In point cloud studies, the point cloud segmentation of individual plants to organs directly determines the accuracy of organ-level phenotype estimation and the reliability of the 3D plant reconstruction. However, highly accurate, automatic, and robust point cloud segmentation approaches for plants are unavailable. Thus, the high-throughput segmentation of many shoots is challenging. Although deep learning can feasibly solve this issue, software tools for 3D point cloud annotation to construct the training dataset are lacking. Results We propose a top-to-down point cloud segmentation algorithm using optimal transportation distance for maize shoots. We apply our point cloud annotation toolkit for maize shoots, Label3DMaize, to achieve semi-automatic point cloud segmentation and annotation of maize shoots at different growth stages, through a series of operations, including stem segmentation, coarse segmentation, fine segmentation, and sample-based segmentation. The toolkit takes ∼4–10 minutes to segment a maize shoot and consumes 10–20% of the total time if only coarse segmentation is required. Fine segmentation is more detailed than coarse segmentation, especially at the organ connection regions. The accuracy of coarse segmentation can reach 97.2% that of fine segmentation. Conclusion Label3DMaize integrates point cloud segmentation algorithms and manual interactive operations, realizing semi-automatic point cloud segmentation of maize shoots at different growth stages. The toolkit provides a practical data annotation tool for further online segmentation research based on deep learning and is expected to promote automatic point cloud processing of various plants.

scholarly journals Multi-Dimensional Underwater Point Cloud Detection Based on Deep Learning

Sensors ◽  
2021 ◽  
Vol 21 (3) ◽  
pp. 884
Author(s):  
Chia-Ming Tsai ◽  
Yi-Horng Lai ◽  
Yung-Da Sun ◽  
Yu-Jen Chung ◽  
Jau-Woei Perng
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Three Dimensional ◽  
Point Clouds ◽  
Training Data ◽  
Network Architectures ◽  
Point Cloud Data ◽  
Data Types ◽  
Raw Data ◽  
Cloud Data

Numerous sensors can obtain images or point cloud data on land, however, the rapid attenuation of electromagnetic signals and the lack of light in water have been observed to restrict sensing functions. This study expands the utilization of two- and three-dimensional detection technologies in underwater applications to detect abandoned tires. A three-dimensional acoustic sensor, the BV5000, is used in this study to collect underwater point cloud data. Some pre-processing steps are proposed to remove noise and the seabed from raw data. Point clouds are then processed to obtain two data types: a 2D image and a 3D point cloud. Deep learning methods with different dimensions are used to train the models. In the two-dimensional method, the point cloud is transferred into a bird’s eye view image. The Faster R-CNN and YOLOv3 network architectures are used to detect tires. Meanwhile, in the three-dimensional method, the point cloud associated with a tire is cut out from the raw data and is used as training data. The PointNet and PointConv network architectures are then used for tire classification. The results show that both approaches provide good accuracy.

scholarly journals Voxel-based 3D Point Cloud Semantic Segmentation: Unsupervised Geometric and Relationship Featuring vs Deep Learning Methods

2019 ◽  
Vol 8 (5) ◽  
pp. 213 ◽  
Author(s):  
Florent Poux ◽  
Roland Billen
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Semantic Representation ◽  
Structural Connectivity ◽  
Three Dimensional ◽  
Strong Support ◽  
Semantic Segmentation ◽  
Point Clouds ◽  
Learning Methods ◽  
Cloud Data

Automation in point cloud data processing is central in knowledge discovery within decision-making systems. The definition of relevant features is often key for segmentation and classification, with automated workflows presenting the main challenges. In this paper, we propose a voxel-based feature engineering that better characterize point clusters and provide strong support to supervised or unsupervised classification. We provide different feature generalization levels to permit interoperable frameworks. First, we recommend a shape-based feature set (SF1) that only leverages the raw X, Y, Z attributes of any point cloud. Afterwards, we derive relationship and topology between voxel entities to obtain a three-dimensional (3D) structural connectivity feature set (SF2). Finally, we provide a knowledge-based decision tree to permit infrastructure-related classification. We study SF1/SF2 synergy on a new semantic segmentation framework for the constitution of a higher semantic representation of point clouds in relevant clusters. Finally, we benchmark the approach against novel and best-performing deep-learning methods while using the full S3DIS dataset. We highlight good performances, easy-integration, and high F1-score (> 85%) for planar-dominant classes that are comparable to state-of-the-art deep learning.

scholarly journals SEGMENTATION OF LARGE UNSTRUCTURED POINT CLOUDS USING OCTREE-BASED REGION GROWING AND CONDITIONAL RANDOM FIELDS

2017 ◽  
Vol XLII-2/W8 ◽  
pp. 25-30 ◽  
Author(s):  
M. Bassier ◽  
M. Bonduel ◽  
B. Van Genechten ◽  
M. Vergauwen
Keyword(s):  
Random Fields ◽  
Point Cloud ◽  
Conditional Random Fields ◽  
Conditional Random Field ◽  
Region Growing ◽  
Point Clouds ◽  
Single Type ◽  
Point Cloud Data ◽  
Cloud Data ◽  
Point Cloud Segmentation

Point cloud segmentation is a crucial step in scene understanding and interpretation. The goal is to decompose the initial data into sets of workable clusters with similar properties. Additionally, it is a key aspect in the automated procedure from point cloud data to BIM. Current approaches typically only segment a single type of primitive such as planes or cylinders. Also, current algorithms suffer from oversegmenting the data and are often sensor or scene dependent.<br><br> In this work, a method is presented to automatically segment large unstructured point clouds of buildings. More specifically, the segmentation is formulated as a graph optimisation problem. First, the data is oversegmented with a greedy octree-based region growing method. The growing is conditioned on the segmentation of planes as well as smooth surfaces. Next, the candidate clusters are represented by a Conditional Random Field after which the most likely configuration of candidate clusters is computed given a set of local and contextual features. The experiments prove that the used method is a fast and reliable framework for unstructured point cloud segmentation. Processing speeds up to 40,000 points per second are recorded for the region growing. Additionally, the recall and precision of the graph clustering is approximately 80%. Overall, nearly 22% of oversegmentation is reduced by clustering the data. These clusters will be classified and used as a basis for the reconstruction of BIM models.

scholarly journals Panicle-3D: Efficient Phenotyping Tool for Precise Semantic Segmentation of Rice Panicle Point Cloud

2021 ◽  
Vol 2021 ◽  
pp. 1-9
Author(s):  
Liang Gong ◽  
Xiaofeng Du ◽  
Kai Zhu ◽  
Ke Lin ◽  
Qiaojun Lou ◽  
...  
Keyword(s):  
Point Cloud ◽  
Network Models ◽  
Semantic Segmentation ◽  
Spatial Location ◽  
Point Clouds ◽  
Utilization Efficiency ◽  
Data Set ◽  
Cloud Data ◽  
Point Cloud Segmentation ◽  
Segmentation Accuracy

The automated measurement of crop phenotypic parameters is of great significance to the quantitative study of crop growth. The segmentation and classification of crop point cloud help to realize the automation of crop phenotypic parameter measurement. At present, crop spike-shaped point cloud segmentation has problems such as fewer samples, uneven distribution of point clouds, occlusion of stem and spike, disorderly arrangement of point clouds, and lack of targeted network models. The traditional clustering method can realize the segmentation of the plant organ point cloud with relatively independent spatial location, but the accuracy is not acceptable. This paper first builds a desktop-level point cloud scanning apparatus based on a structured-light projection module to facilitate the point cloud acquisition process. Then, the rice ear point cloud was collected, and the rice ear point cloud data set was made. In addition, data argumentation is used to improve sample utilization efficiency and training accuracy. Finally, a 3D point cloud convolutional neural network model called Panicle-3D was designed to achieve better segmentation accuracy. Specifically, the design of Panicle-3D is aimed at the multiscale characteristics of plant organs, combined with the structure of PointConv and long and short jumps, which accelerates the convergence speed of the network and reduces the loss of features in the process of point cloud downsampling. After comparison experiments, the segmentation accuracy of Panicle-3D reaches 93.4%, which is higher than PointNet. Panicle-3D is suitable for other similar crop point cloud segmentation tasks.

scholarly journals Review: Deep Learning on 3D Point Clouds

2020 ◽  
Vol 12 (11) ◽  
pp. 1729 ◽  
Author(s):  
Saifullahi Aminu Bello ◽  
Shangshu Yu ◽  
Cheng Wang ◽  
Jibril Muhmmad Adam ◽  
Jonathan Li
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
General Structure ◽  
Point Clouds ◽  
Autonomous Driving ◽  
Point Cloud Data ◽  
3D Vision ◽  
Cloud Data ◽  
3D Point Clouds ◽  
Learning Techniques

A point cloud is a set of points defined in a 3D metric space. Point clouds have become one of the most significant data formats for 3D representation and are gaining increased popularity as a result of the increased availability of acquisition devices, as well as seeing increased application in areas such as robotics, autonomous driving, and augmented and virtual reality. Deep learning is now the most powerful tool for data processing in computer vision and is becoming the most preferred technique for tasks such as classification, segmentation, and detection. While deep learning techniques are mainly applied to data with a structured grid, the point cloud, on the other hand, is unstructured. The unstructuredness of point clouds makes the use of deep learning for its direct processing very challenging. This paper contains a review of the recent state-of-the-art deep learning techniques, mainly focusing on raw point cloud data. The initial work on deep learning directly with raw point cloud data did not model local regions; therefore, subsequent approaches model local regions through sampling and grouping. More recently, several approaches have been proposed that not only model the local regions but also explore the correlation between points in the local regions. From the survey, we conclude that approaches that model local regions and take into account the correlation between points in the local regions perform better. Contrary to existing reviews, this paper provides a general structure for learning with raw point clouds, and various methods were compared based on the general structure. This work also introduces the popular 3D point cloud benchmark datasets and discusses the application of deep learning in popular 3D vision tasks, including classification, segmentation, and detection.

Supervoxel-Based Region Growing Segmentation for Point Cloud Data

2020 ◽  
pp. 2154007
Author(s):  
Nan Luo ◽  
Yuanyuan Jiang ◽  
Quan Wang
Keyword(s):  
Point Cloud ◽  
High Efficiency ◽  
Region Growing ◽  
Experimental Tests ◽  
Point Clouds ◽  
Spatial Distance ◽  
Local Geometry ◽  
Cloud Data ◽  
Point Cloud Segmentation ◽  
Region Growing Segmentation

Point cloud segmentation is a crucial fundamental step in 3D reconstruction, object recognition and scene understanding. This paper proposes a supervoxel-based point cloud segmentation algorithm in region growing principle to solve the issues of inaccurate boundaries and nonsmooth segments in the existing methods. To begin with, the input point cloud is voxelized and then pre-segmented into sparse supervoxels by flow constrained clustering, considering the spatial distance and local geometry between voxels. Afterwards, plane fitting is applied to the over-segmented supervoxels and seeds for region growing are selected with respect to the fitting residuals. Starting from pruned seed patches, adjacent supervoxels are merged in region growing style to form the final segments, according to the normalized similarity measure that integrates the smoothness and shape constraints of supervoxels. We determine the values of parameters via experimental tests, and the final results show that, by voxelizing and pre-segmenting the point clouds, the proposed algorithm is robust to noises and can obtain smooth segmentation regions with accurate boundaries in high efficiency.

scholarly journals Collapsed Building Detection Using 3D Point Clouds and Deep Learning

2020 ◽  
Vol 12 (24) ◽  
pp. 4057
Author(s):  
Haoyi Xiu ◽  
Takayuki Shinohara ◽  
Masashi Matsuoka ◽  
Munenari Inoguchi ◽  
Ken Kawabe ◽  
...  
Keyword(s):  
Deep Learning ◽  
Damage Detection ◽  
Point Cloud ◽  
Point Clouds ◽  
Building Detection ◽  
Building Roof ◽  
Cloud Data ◽  
Architectural Styles ◽  
Collapsed Buildings ◽  
Collapsed Building

Collapsed buildings should be detected with the highest priority during earthquake emergency response, due to the associated fatality rates. Although deep learning-based damage detection using vertical aerial images can achieve high performance, as depth information cannot be obtained, it is difficult to detect collapsed buildings when their roofs are not heavily damaged. Airborne LiDAR can efficiently obtain the 3D geometries of buildings (in the form of point clouds) and thus has greater potential to detect various collapsed buildings. However, there have been few previous studies on deep learning-based damage detection using point cloud data, due to a lack of large-scale datasets. Therefore, in this paper, we aim to develop a dataset tailored to point cloud-based building damage detection, in order to investigate the potential of point cloud data in collapsed building detection. Two types of building data are created: building roof and building patch, which contains the building and its surroundings. Comprehensive experiments are conducted under various data availability scenarios (pre–post-building patch, post-building roof, and post-building patch) with varying reference data. The pre–post scenario tries to detect damage using pre-event and post-event data, whereas post-building patch and roof only use post-event data. Damage detection is implemented using both basic and modern 3D point cloud-based deep learning algorithms. To adapt a single-input network, which can only accept one building’s data for a prediction, to the pre–post (double-input) scenario, a general extension framework is proposed. Moreover, a simple visual explanation method is proposed, in order to conduct sensitivity analyses for validating the reliability of model decisions under the post-only scenario. Finally, the generalization ability of the proposed approach is tested using buildings with different architectural styles acquired by a distinct sensor. The results show that point cloud-based methods can achieve high accuracy and are robust under training data reduction. The sensitivity analysis reveals that the trained models are able to locate roof deformations precisely, but have difficulty recognizing global damage, such as that relating to the roof inclination. Additionally, it is revealed that the model decisions are overly dependent on debris-like objects when surroundings information is available, which leads to misclassifications. By training on the developed dataset, the model can achieve moderate accuracy on another dataset with different architectural styles without additional training.

scholarly journals AN EXTRACTION METHOD FOR ROOF POINT CLOUD OF ANCIENT BUILDING USING DEEP LEARNING FRAMEWORK

2021 ◽  
Vol XLVI-M-1-2021 ◽  
pp. 321-327
Author(s):  
Y. Ji ◽  
Y. Dong ◽  
M. Hou ◽  
Y. Qi ◽  
A. Li
Keyword(s):  
Deep Learning ◽  
Point Cloud ◽  
Learning Algorithm ◽  
Point Clouds ◽  
Structural Features ◽  
Real Point ◽  
Labor Costs ◽  
Dynamic Graph ◽  
Cloud Data ◽  
Roof Structure

Abstract. Chinese ancient architecture is a valuable heritage wealth, especially for roof that reflects the construction age, structural features and cultural connotation. Point cloud data, as a flexible representation with characteristics of fast, precise, non-contact, plays a crucial role in a variety of applications for ancient architectural heritage, such as 3D fine reconstruction, HBIM, disaster monitoring etc. However, there are still many limitations in data editing tasks that need to be worked out manually, which is time-consuming, labor-intensive and error-prone. In recent years, the theoretical advance on deep learning has stimulated the development of various domains, and digital heritage is not in exception. Whenever, deep learning algorithm need to consume a huge amount of labeled date to achieve the purpose for segmentation, resulting a actuality that high labor costs also be acquired. In this paper, inspired by the architectural style similarity between mimetic model and real building, we proposed a method supported by deep learning, which aims to give a solution for the point cloud automatic extraction of roof structure. Firstly, to generate real point cloud, Baoguang Temple, unmanned Aerial Vehicle (UAV) is presented to obtain image collections that are subsequently processed by reconstruction technology. Secondly, a modified Dynamic Graph Convolutional Neural Network (DGCNN) which can learn local features with taking advantage of an edge attention convolution is trained using simulated data and additional attributes of geometric attributes. The mimetic data is sampled from 3DMAX model surface. Finally, we try to extract roof structure of ancient building from real point clouds scenes utilizing the trained model. The experimental results show that the proposed method can extract the rooftop structure from real scene of Baoguang, which illustrates not only effectiveness of approach but also a fact that the simulated source perform potential value when real point cloud datasets are scarce.

scholarly journals Building Extraction from Airborne Multi-Spectral LiDAR Point Clouds Based on Graph Geometric Moments Convolutional Neural Networks

2020 ◽  
Vol 12 (19) ◽  
pp. 3186 ◽  
Author(s):  
Dilong Li ◽  
Xin Shen ◽  
Yongtao Yu ◽  
Haiyan Guan ◽  
Jonathan Li ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Point Cloud ◽  
Point Clouds ◽  
Hierarchical Architecture ◽  
Building Extraction ◽  
Cloud Data ◽  
Geometric Moments ◽  
Point Cloud Segmentation ◽  
Previous State

Building extraction has attracted much attentions for decades as a prerequisite for many applications and is still a challenging topic in the field of photogrammetry and remote sensing. Due to the lack of spectral information, massive data processing, and approach universality, building extraction from point clouds is still a thorny and challenging problem. In this paper, a novel deep-learning-based framework is proposed for building extraction from point cloud data. Specifically, first, a sample generation method is proposed to split the raw preprocessed multi-spectral light detection and ranging (LiDAR) data into numerous samples, which are directly fed into convolutional neural networks and completely cover the original inputs. Then, a graph geometric moments (GGM) convolution is proposed to encode the local geometric structure of point sets. In addition, a hierarchical architecture equipped with GGM convolution, called GGM convolutional neural networks, is proposed to train and recognize building points. Finally, the test scenes with varying sizes can be fed into the framework and obtain a point-wise extraction result. We evaluate the proposed framework and methods on the airborne multi-spectral LiDAR point clouds collected by an Optech Titan system. Compared with previous state-of-the-art networks, which are designed for point cloud segmentation, our method achieves the best performance with a correctness of 95.1%, a completeness of 93.7%, an F-measure of 94.4%, and an intersection over union (IoU) of 89.5% on two test areas. The experimental results confirm the effectiveness and efficiency of the proposed framework and methods.

Sign in / Sign up

Export Citation Format

Share Document