scholarly journals An Investigation of Surrogate Models for Efficient Performance-Based Decoding of 3D Point Clouds

2019 ◽  
Vol 141 (12) ◽  
Author(s):  
James D. Cunningham ◽  
Timothy W. Simpson ◽  
Conrad S. Tucker
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Surrogate Model ◽  
Surrogate Modeling ◽  
Point Clouds ◽  
Absolute Error ◽  
3D Models ◽  
Test Case ◽  
3D Point Clouds ◽  
Comparable Accuracy

Abstract This work investigates surrogate modeling techniques for learning to approximate a computationally expensive function evaluation of 3D models. While in the past, 3D point clouds have been a data format that is too high dimensional for surrogate modeling, by leveraging advances in 3D object autoencoding neural networks, these point clouds can be mapped to a one-dimensional latent space. This leads to the fundamental research question: what surrogate modeling technique is most suitable for learning relationships between the 3D geometric features of the objects captured in the encoded latent vector and the physical phenomena captured in the evaluation software? Radial basis functions (RBFs), Kriging, and shallow 1D analogs of popular deep 2D image classification neural networks are investigated in this work. We find the nonintuitive result that departing from neural networks to decode latent representations of 3D objects into performance predictions is far more efficient than using a neural network decoder. In test cases using datasets of aircraft and watercraft 3D models, the non-neural network surrogate models achieve comparable accuracy to the neural network models. We find that an RBF surrogate model is able to approximate the lift and drag coefficients of 234 aircraft models with a mean absolute error of 1.97 × 10−3 and trains in only 3 seconds. Furthermore, the RBF surrogate model is able to rank a set of designs with an average percentile error of less than 8%. In comparison, a 1D ResNet achieves an average absolute error of 1.35 × 103 in 38 min for the same test case. We validate the comparable accuracy of the four techniques through a test case involving 214 3D watercraft models, but we also find that the distribution of the performance values of the data, in particular the presence of many outliers, has a significant negative impact on accuracy. These results contradict a common perception of neural networks as an efficient “one-size-fits-all” solution for learning black-box functions and suggests that even within systems that utilize multiple neural networks, potentially more efficient alternatives should be considered for each network in the system. Depending on the required accuracy of the application, this surrogate modeling approach could be used to approximate an expensive simulation software, or if the tolerance for error is low, it serves as a first pass which can narrow down the number of candidate designs to be analyzed more thoroughly.

scholarly journals Convolutional Neural Network for Extracting 3D Point Clouds of Fibrous Web From Multi-Focus Images

IEEE Access ◽  
2020 ◽  
Vol 8 ◽  
pp. 87857-87869
Author(s):  
Jue Hou ◽  
Wenbin Ouyang ◽  
Bugao Xu ◽  
Rongwu Wang
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
Point Clouds ◽  
3D Point Clouds

scholarly journals Enhancing protein backbone angle prediction by using simpler models of deep neural networks

2020 ◽  
Vol 10 (1) ◽  
Author(s):  
Fereshteh Mataeimoghadam ◽  
M. A. Hakim Newton ◽  
Abdollah Dehzangi ◽  
Abdul Karim ◽  
B. Jayaram ◽  
...  
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Structure Prediction ◽  
Protein Structures ◽  
Absolute Error ◽  
Grand Challenge ◽  
Protein Backbone ◽  
The Neural Network ◽  
Benchmark Datasets ◽  
The Neural Networks

Abstract Protein structure prediction is a grand challenge. Prediction of protein structures via the representations using backbone dihedral angles has recently achieved significant progress along with the on-going surge of deep neural network (DNN) research in general. However, we observe that in the protein backbone angle prediction research, there is an overall trend to employ more and more complex neural networks and then to throw more and more features to the neural networks. While more features might add more predictive power to the neural network, we argue that redundant features could rather clutter the scenario and more complex neural networks then just could counterbalance the noise. From artificial intelligence and machine learning perspectives, problem representations and solution approaches do mutually interact and thus affect performance. We also argue that comparatively simpler predictors can more easily be reconstructed than the more complex ones. With these arguments in mind, we present a deep learning method named Simpler Angle Predictor (SAP) to train simpler DNN models that enhance protein backbone angle prediction. We then empirically show that SAP can significantly outperform existing state-of-the-art methods on well-known benchmark datasets: for some types of angles, the differences are 6–8 in terms of mean absolute error (MAE). The SAP program along with its data is available from the website https://gitlab.com/mahnewton/sap.

scholarly journals Three-Dimensional Reconstruction of Structural Surface Model of Heritage Bridges Using UAV-Based Photogrammetric Point Clouds

2019 ◽  
Vol 11 (10) ◽  
pp. 1204 ◽  
Author(s):  
Yue Pan ◽  
Yiqing Dong ◽  
Dalei Wang ◽  
Airong Chen ◽  
Zhen Ye
Keyword(s):  
Classification Tree ◽  
Three Dimensional ◽  
Point Clouds ◽  
3D Models ◽  
Surface Model ◽  
Reconstruction Method ◽  
3D Point Clouds ◽  
Surface Models ◽  
Structural Surface ◽  
Better Than

Three-dimensional (3D) digital technology is essential to the maintenance and monitoring of cultural heritage sites. In the field of bridge engineering, 3D models generated from point clouds of existing bridges is drawing increasing attention. Currently, the widespread use of the unmanned aerial vehicle (UAV) provides a practical solution for generating 3D point clouds as well as models, which can drastically reduce the manual effort and cost involved. In this study, we present a semi-automated framework for generating structural surface models of heritage bridges. To be specific, we propose to tackle this challenge via a novel top-down method for segmenting main bridge components, combined with rule-based classification, to produce labeled 3D models from UAV photogrammetric point clouds. The point clouds of the heritage bridge are generated from the captured UAV images through the structure-from-motion workflow. A segmentation method is developed based on the supervoxel structure and global graph optimization, which can effectively separate bridge components based on geometric features. Then, recognition by the use of a classification tree and bridge geometry is utilized to recognize different structural elements from the obtained segments. Finally, surface modeling is conducted to generate surface models of the recognized elements. Experiments using two bridges in China demonstrate the potential of the presented structural model reconstruction method using UAV photogrammetry and point cloud processing in 3D digital documentation of heritage bridges. By using given markers, the reconstruction error of point clouds can be as small as 0.4%. Moreover, the precision and recall of segmentation results using testing date are better than 0.8, and a recognition accuracy better than 0.8 is achieved.

scholarly journals ON SELECTING IMAGES FROM AN UNAIMED VIDEO STREAM FOR PHOTOGRAMMETRIC MODELLING

2020 ◽  
Vol V-2-2020 ◽  
pp. 389-394
Author(s):  
P. Rönnholm ◽  
M. T. Vaaja ◽  
H. Kauhanen ◽  
T. Klockars
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
3D Models ◽  
Video Stream ◽  
Test Case ◽  
Data Sets ◽  
Image Block ◽  
Imaging Geometry ◽  
Good Imaging ◽  
Voxel Representation

Abstract. In this paper, we illustrate how convolutional neural networks and voxel-based processing together with voxel visualizations can be utilized for the selection of unaimed images for a photogrammetric image block. Our research included the detection of an ear from images with a convolutional neural network, computation of image orientations with a structure-from-motion algorithm, visualization of camera locations in a voxel representation to detect the goodness of the imaging geometry, rejection of unnecessary images with an XYZ buffer, the creation of 3D models in two different example cases, and the comparison of resulting 3D models. Two test data sets were taken of an ear with the video recorder of a mobile phone. In the first test case, a special emphasis was taken to ensure good imaging geometry. On the contrary, in the second test case the trajectory was limited to approximately horizontal movement, leading to poor imaging geometry. A convolutional neural network together with an XYZ buffer managed to select a useful set of images for the photogrammetric 3D measuring phase. The voxel representation well illustrated the imaging geometry and has potential for early detection where data is suitable for photogrammetric modelling. The comparison of 3D models revealed that the model from poor imaging geometry was noisy and flattened. The results emphasize the importance of good imaging geometry.

scholarly journals APPLICATIONS OF PANORAMIC IMAGES: FROM 720° PANORAMA TO INTERIOR 3D MODELS OF AUGMENTED REALITY

2015 ◽  
Vol XL-4/W5 ◽  
pp. 189-192 ◽  
Author(s):  
I.-C. Lee ◽  
F. Tsai
Keyword(s):  
Augmented Reality ◽  
Focal Length ◽  
Three Dimensional ◽  
Point Clouds ◽  
3D Models ◽  
Exterior Orientation ◽  
3D Point Clouds ◽  
Panoramic Images ◽  
Main Station ◽  
Orientation Parameters

A series of panoramic images are usually used to generate a 720° panorama image. Although panoramic images are typically used for establishing tour guiding systems, in this research, we demonstrate the potential of using panoramic images acquired from multiple sites to create not only 720° panorama, but also three-dimensional (3D) point clouds and 3D indoor models. Since 3D modeling is one of the goals of this research, the location of the panoramic sites needed to be carefully planned in order to maintain a robust result for close-range photogrammetry. After the images are acquired, panoramic images are processed into 720° panoramas, and these panoramas which can be used directly as panorama guiding systems or other applications. <br><br> In addition to these straightforward applications, interior orientation parameters can also be estimated while generating 720° panorama. These parameters are focal length, principle point, and lens radial distortion. The panoramic images can then be processed with closerange photogrammetry procedures to extract the exterior orientation parameters and generate 3D point clouds. In this research, VisaulSFM, a structure from motion software is used to estimate the exterior orientation, and CMVS toolkit is used to generate 3D point clouds. Next, the 3D point clouds are used as references to create building interior models. In this research, Trimble Sketchup was used to build the model, and the 3D point cloud was added to the determining of locations of building objects using plane finding procedure. In the texturing process, the panorama images are used as the data source for creating model textures. This 3D indoor model was used as an Augmented Reality model replacing a guide map or a floor plan commonly used in an on-line touring guide system. <br><br> The 3D indoor model generating procedure has been utilized in two research projects: a cultural heritage site at Kinmen, and Taipei Main Station pedestrian zone guidance and navigation system. The results presented in this paper demonstrate the potential of using panoramic images to generate 3D point clouds and 3D models. However, it is currently a manual and labor-intensive process. A research is being carried out to Increase the degree of automation of these procedures.

scholarly journals PCEDNet: A Lightweight Neural Network for Fast and Interactive Edge Detection in 3D Point Clouds

2022 ◽  
Vol 41 (1) ◽  
pp. 1-21
Author(s):  
Chems-Eddine Himeur ◽  
Thibault Lejemble ◽  
Thomas Pellegrini ◽  
Mathias Paulin ◽  
Loic Barthe ◽  
...  
Keyword(s):  
Neural Network ◽  
Network Architecture ◽  
Point Clouds ◽  
Scale Space ◽  
Neural Network Architecture ◽  
Multi Scale ◽  
3D Point Clouds ◽  
Space Matrix ◽  
Processing Point

In recent years, Convolutional Neural Networks (CNN) have proven to be efficient analysis tools for processing point clouds, e.g., for reconstruction, segmentation, and classification. In this article, we focus on the classification of edges in point clouds, where both edges and their surrounding are described. We propose a new parameterization adding to each point a set of differential information on its surrounding shape reconstructed at different scales. These parameters, stored in a Scale-Space Matrix (SSM) , provide a well-suited information from which an adequate neural network can learn the description of edges and use it to efficiently detect them in acquired point clouds. After successfully applying a multi-scale CNN on SSMs for the efficient classification of edges and their neighborhood, we propose a new lightweight neural network architecture outperforming the CNN in learning time, processing time, and classification capabilities. Our architecture is compact, requires small learning sets, is very fast to train, and classifies millions of points in seconds.

scholarly journals Urban Feature Extraction from Merged Airborne LiDAR Data and Digital Camera Data

2021 ◽  
Vol 7 (2) ◽  
pp. 57-74
Author(s):  
Lamyaa Gamal EL-Deen Taha ◽  
A. I. Ramzi ◽  
A. Syarawi ◽  
A. Bekheet
Keyword(s):  
Neural Network ◽  
Digital Camera ◽  
Point Clouds ◽  
Airborne Lidar ◽  
Surface Model ◽  
Digital Surface Model ◽  
3D Point Clouds ◽  
Digital Elevation ◽  
Surface Models ◽  
Elevation Model

Until recently, the most highly accurate digital surface models were obtained from airborne lidar. With the development of a new generation of large format digital photogrammetric aerial camera, a fully digital photogrammetric workflow became possible. Digital airborne images are sources for elevation extraction and orthophoto generation. This research concerned with the generation of digital surface models and orthophotos as applications from high-resolution images.  In this research, the following steps were performed. A Benchmark data of LIDAR and digital aerial camera have been used.  Firstly, image orientation, AT have been performed. Then the automatic digital surface model DSM generation has been produced from the digital aerial camera. Thirdly true digital ortho has been generated from the digital aerial camera also orthoimage will be generated using LIDAR digital elevation model (DSM). Leica Photogrammetric Suite (LPS) module of Erdsa Imagine 2014 software was utilized for processing. Then the resulted orthoimages from both techniques were mosaicked. The results show that automatic digital surface model DSM that been produced from digital aerial camera method has very high dense photogrammetric 3D point clouds compared to the LIDAR 3D point clouds. It was found that the true orthoimage produced from the second approach is better than the true orthoimage produced from the first approach. The five approaches were tested for classification of the best-orthorectified image mosaic using subpixel based (neural network) and pixel-based ( minimum distance and maximum likelihood).Multicues were extracted such as texture(entropy-mean),Digital elevation model, Digital surface model ,normalized digital surface model (nDSM) and intensity image. The contributions of the individual cues used in the classification have been evaluated. It was found that the best cue integration is intensity (pan) +nDSM+ entropy followed by intensity (pan) +nDSM+mean then intensity image +mean+ entropy after that DSM )image and two texture measures (mean and entropy) followed by the colour image. The integration with height data increases the accuracy. Also, it was found that the integration with entropy texture increases the accuracy. Resulted in fifteen cases of classification it was found that maximum likelihood classifier is the best followed by minimum distance then neural network classifier. We attribute this to the fine resolution of the digital camera image. Subpixel classifier (neural network) is not suitable for classifying aerial digital camera images. 

Investigation of Applying Physics Informed Neural Networks (PINN) and Variants on 2D Aerodynamics Problems

2020 ◽  
Author(s):  
Wee-Beng Tay ◽  
Murali Damodaran ◽  
Zhi-Da Teh ◽  
Rahul Halder
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Network ◽  
Training Data ◽  
Batch Size ◽  
Test Case ◽  
Good Prediction ◽  
Time Varying ◽  
Artificial Neural ◽  
Artificial Neural Network Ann

Abstract Investigation of applying physics informed neural networks on the test case involving flow past Converging-Diverging (CD) Nozzle has been investigated. Both Artificial Neural Network (ANN) and Physics Informed Neural Network (PINN) are used to do the training and prediction. Results show that Artificial Neural Network (ANN) by itself is already able to give relatively good prediction. With the addition of PINN, the error reduces even more, although by only a relatively small amount. This is perhaps due to the already good prediction. The effects of batch size, training iteration and number of epochs on the prediction accuracy have already been tested. It is found that increasing batch size improves the prediction. On the other hand, increasing the training iteration may give poorer prediction due to overfitting. Lastly, in general, increasing epochs reduces the error. More investigations should be done in the future to further reduce the error while at the same time using less training data. More complicated cases with time varying results should also be included. Extrapolation of the results using PINN can also be tested.

scholarly journals A Self-Assembly Portable Mobile Mapping System for Archeological Reconstruction Based on VSLAM-Photogrammetric Algorithm

Sensors ◽  
2019 ◽  
Vol 19 (18) ◽  
pp. 3952 ◽  
Author(s):  
* ◽  
*
Keyword(s):  
Self Assembly ◽  
Average Distance ◽  
Low Cost ◽  
Three Dimensional ◽  
Point Clouds ◽  
3D Models ◽  
Data Capture ◽  
Information Modeling ◽  
3D Point Clouds ◽  
Mean Square Errors

Three Dimensional (3D) models are widely used in clinical applications, geosciences, cultural heritage preservation, and engineering; this, together with new emerging needs such as building information modeling (BIM) develop new data capture techniques and devices with a low cost and reduced learning curve that allow for non-specialized users to employ it. This paper presents a simple, self-assembly device for 3D point clouds data capture with an estimated base price under €2500; furthermore, a workflow for the calculations is described that includes a Visual SLAM-photogrammetric threaded algorithm that has been implemented in C++. Another purpose of this work is to validate the proposed system in BIM working environments. To achieve it, in outdoor tests, several 3D point clouds were obtained and the coordinates of 40 points were obtained by means of this device, with data capture distances ranging between 5 to 20 m. Subsequently, those were compared to the coordinates of the same targets measured by a total station. The Euclidean average distance errors and root mean square errors (RMSEs) ranging between 12–46 mm and 8–33 mm respectively, depending on the data capture distance (5–20 m). Furthermore, the proposed system was compared with a commonly used photogrammetric methodology based on Agisoft Metashape software. The results obtained demonstrate that the proposed system satisfies (in each case) the tolerances of ‘level 1’ (51 mm) and ‘level 2’ (13 mm) for point cloud acquisition in urban design and historic documentation, according to the BIM Guide for 3D Imaging (U.S. General Services).

scholarly journals A 3D Point Cloud Filtering Method for Leaves Based on Manifold Distance and Normal Estimation

2019 ◽  
Vol 11 (2) ◽  
pp. 198 ◽  
Author(s):  
Chunhua Hu ◽  
Zhou Pan ◽  
Pingping Li
Keyword(s):  
Leaf Area ◽  
Point Cloud ◽  
Mean Absolute Error ◽  
Point Clouds ◽  
Absolute Error ◽  
Area Measurement ◽  
3D Point Cloud ◽  
Filtering Method ◽  
Normal Estimation ◽  
3D Point Clouds

Leaves are used extensively as an indicator in research on tree growth. Leaf area, as one of the most important index in leaf morphology, is also a comprehensive growth index for evaluating the effects of environmental factors. When scanning tree surfaces using a 3D laser scanner, the scanned point cloud data usually contain many outliers and noise. These outliers can be clusters or sparse points, whereas the noise is usually non-isolated but exhibits different attributes from valid points. In this study, a 3D point cloud filtering method for leaves based on manifold distance and normal estimation is proposed. First, leaf was extracted from the tree point cloud and initial clustering was performed as the preprocessing step. Second, outlier clusters filtering and outlier points filtering were successively performed using a manifold distance and truncation method. Third, noise points in each cluster were filtered based on the local surface normal estimation. The 3D reconstruction results of leaves after applying the proposed filtering method prove that this method outperforms other classic filtering methods. Comparisons of leaf areas with real values and area assessments of the mean absolute error (MAE) and mean absolute error percent (MAE%) for leaves in different levels were also conducted. The root mean square error (RMSE) for leaf area was 2.49 cm2. The MAE values for small leaves, medium leaves and large leaves were 0.92 cm2, 1.05 cm2 and 3.39 cm2, respectively, with corresponding MAE% values of 10.63, 4.83 and 3.8. These results demonstrate that the method proposed can be used to filter outliers and noise for 3D point clouds of leaves and improve 3D leaf visualization authenticity and leaf area measurement accuracy.

Sign in / Sign up

Export Citation Format

Share Document