scholarly journals PointNet and geometric reasoning for detection of grape vines from single frame RGB-D data in outdoor conditions

2020 ◽  
Vol 1 ◽  
pp. 6
Author(s):  
Polina Kurtser ◽  
Ola Ringdahl ◽  
Nati Rotstein ◽  
Henrik Andreasson
Keyword(s):  
Point Clouds ◽  
Geometric Reasoning ◽  
Added Value ◽  
Color Contrast ◽  
Significant Drop ◽  
Grape Vine ◽  
Depth Cameras ◽  
Set Size ◽  
Agricultural Robotics ◽  
Major Bottleneck

In this paper we present the usage of PointNet, a deep neural network that consumes raw un-ordered point clouds, for detection of grape vine clusters in outdoor conditions. We investigate the added value of feeding the detection network with both RGB and depth, contradictory to common practice in agricultural robotics of relying on RGB only. A total of 5057 pointclouds (1033 manually annotated and 4024 annotated using geometric reasoning) were collected in a field experiment conducted in outdoor conditions on 9 grape vines and 5 plants. The detection results show overall accuracy of 91% (average class accuracy of 74%, precision 53% recall 48%) for RGBXYZ data and a significant drop in recall for RGB or XYZ data only. These results suggest the usage of depth cameras for vision in agricultural robotics is crucial for crops where the color contrast between the crop and the background is complex. The results also suggest geometric reasoning can be used for increased training set size, a major bottleneck in the development of agricultural vision systems.

A depth camera-based full-body reconstruction method for body pose training in occupational safety

2020 ◽  
Vol 64 (1) ◽  
pp. 1541-1545
Author(s):  
Li Li ◽  
Ken Chen ◽  
Karen Chen ◽  
Xu Xu*
Keyword(s):  
Occupational Injuries ◽  
Technological Development ◽  
Point Clouds ◽  
Incidence Rates ◽  
Multiple Point ◽  
Reconstruction Method ◽  
Safety Education ◽  
Depth Cameras ◽  
Work Related ◽  
Full Body

Occupational injuries have high incidence rates across various industries. Safety education is a key component to effectively reduce work-related injuries. Posture training for work safety is widely adopted to increase the awareness of unsafe movements at work and to evaluate workers to minimize work-related musculoskeletal stresses. However, existing one-size-fits-all pamphlet-based posture training is facing challenges in its effectiveness. In recent years, the substantial technological development in virtual reality (VR) and augmented reality (AR) has made immersive and personalized education possible. For VR/AR-assisted posture training, full-body reconstruction from multiple point clouds is the key step. In this study, we propose a fast and coarse method to reconstruct the full-body pose of safety instructors using multiple low-cost depth cameras. The reconstructed body images from depth cameras are registered through iterative closet point algorithm. The reconstructed full-body pose can be further rendered in VR/AR environments for next-generation safety education.

scholarly journals The Accuracy Comparison of Three Simultaneous Localization and Mapping (SLAM)-Based Indoor Mapping Technologies

Sensors ◽  
2018 ◽  
Vol 18 (10) ◽  
pp. 3228 ◽  
Author(s):  
Yuwei Chen ◽  
Jian Tang ◽  
Changhui Jiang ◽  
Lingli Zhu ◽  
Matti Lehtomäki ◽  
...  
Keyword(s):  
Indoor Environment ◽  
Simultaneous Localization And Mapping ◽  
Point Clouds ◽  
Medium Size ◽  
Accuracy Evaluation ◽  
Depth Cameras ◽  
Localization And Mapping ◽  
Mapping System ◽  
Small Footprint ◽  
Mapping Systems

The growing interest and the market for indoor Location Based Service (LBS) have been drivers for a huge demand for building data and reconstructing and updating of indoor maps in recent years. The traditional static surveying and mapping methods can’t meet the requirements for accuracy, efficiency and productivity in a complicated indoor environment. Utilizing a Simultaneous Localization and Mapping (SLAM)-based mapping system with ranging and/or camera sensors providing point cloud data for the maps is an auspicious alternative to solve such challenges. There are various kinds of implementations with different sensors, for instance LiDAR, depth cameras, event cameras, etc. Due to the different budgets, the hardware investments and the accuracy requirements of indoor maps are diverse. However, limited studies on evaluation of these mapping systems are available to offer a guideline of appropriate hardware selection. In this paper we try to characterize them and provide some extensive references for SLAM or mapping system selection for different applications. Two different indoor scenes (a L shaped corridor and an open style library) were selected to review and compare three different mapping systems, namely: (1) a commercial Matterport system equipped with depth cameras; (2) SLAMMER: a high accuracy small footprint LiDAR with a fusion of hector-slam and graph-slam approaches; and (3) NAVIS: a low-cost large footprint LiDAR with Improved Maximum Likelihood Estimation (IMLE) algorithm developed by the Finnish Geospatial Research Institute (FGI). Firstly, an L shaped corridor (2nd floor of FGI) with approximately 80 m length was selected as the testing field for Matterport testing. Due to the lack of quantitative evaluation of Matterport indoor mapping performance, we attempted to characterize the pros and cons of the system by carrying out six field tests with different settings. The results showed that the mapping trajectory would influence the final mapping results and therefore, there was optimal Matterport configuration for better indoor mapping results. Secondly, a medium-size indoor environment (the FGI open library) was selected for evaluation of the mapping accuracy of these three indoor mapping technologies: SLAMMER, NAVIS and Matterport. Indoor referenced maps were collected with a small footprint Terrestrial Laser Scanner (TLS) and using spherical registration targets. The 2D indoor maps generated by these three mapping technologies were assessed by comparing them with the reference 2D map for accuracy evaluation; two feature selection methods were also utilized for the evaluation: interactive selection and minimum bounding rectangles (MBRs) selection. The mapping RMS errors of SLAMMER, NAVIS and Matterport were 2.0 cm, 3.9 cm and 4.4 cm, respectively, for the interactively selected features, and the corresponding values using MBR features were 1.7 cm, 3.2 cm and 4.7 cm. The corresponding detection rates for the feature points were 100%, 98.9%, 92.3% for the interactive selected features and 100%, 97.3% and 94.7% for the automated processing. The results indicated that the accuracy of all the evaluated systems could generate indoor map at centimeter-level, but also variation of the density and quality of collected point clouds determined the applicability of a system into a specific LBS.

Towards Remote Teleoperation of a Semi-Autonomous Mobile Manipulator System in Machine Tending Tasks

2019 ◽  
Author(s):  
Vivek Annem ◽  
Pradeep Rajendran ◽  
Shantanu Thakar ◽  
Satyandra K. Gupta
Keyword(s):  
Material Handling ◽  
Point Clouds ◽  
Mobile Manipulator ◽  
Depth Cameras ◽  
Level Of Automation ◽  
Volume Production ◽  
Manufacturing Applications ◽  
High Level ◽  
The Given ◽  
Mobile Base

Abstract Increasing the level of automation in material handling tasks in small volume production operations can improve human productivity and overall manufacturing system performance. In this paper, we present a teleoperated mobile manipulator system that can be used for tending machines and transporting parts in manufacturing applications. The remotely located human operator can interact with the semi-autonomous mobile manipulator by giving it high level instructions. We have incorporated several sensors on the system to ensure safe teleoperation where the operator gives only high level motion goals to the mobile manipulator, such as waypoints for mobile base motion, and interactive marker poses for the manipulator motion. The point clouds from multiple depth cameras are used for mapping the environment. The robot plans for autonomous motions between the given way-points ensuring that the resulting motions are collision-free. We have conducted case studies with two different types of parts to be extracted from a 3D printer. The system is tested by multiple users. They were successful in completing tasks in a reasonable amount of time using our interface.

scholarly journals Virtual Valcamonica: Collaborative Exploration of Prehistoric Petroglyphs and Their Surrounding Environment in Multi-User Virtual Reality

2018 ◽  
Vol 26 (3) ◽  
pp. 297-321 ◽  
Author(s):  
Alexander Kulik ◽  
André Kunert ◽  
Stephan Beck ◽  
Carl-Feofan Matthes ◽  
Andre Schollmeyer ◽  
...  
Keyword(s):  
Virtual Reality ◽  
Visual Analysis ◽  
Rock Art ◽  
Aggregate Size ◽  
Point Clouds ◽  
Added Value ◽  
3D Analysis ◽  
Interaction Techniques ◽  
3D Interaction ◽  
3D Point Clouds

In this article, we present a novel, multi-user, virtual reality environment for the interactive, collaborative 3D analysis of large 3D scans and the technical advancements that were necessary to build it: a multi-view rendering system for large 3D point clouds, a suitable display infrastructure, and a suite of collaborative 3D interaction techniques. The cultural heritage site of Valcamonica in Italy with its large collection of prehistoric rock-art served as an exemplary use case for evaluation. The results show that our output-sensitive level-of-detail rendering system is capable of visualizing a 3D dataset with an aggregate size of more than 14 billion points at interactive frame rates. The system design in this exemplar application results from close exchange with a small group of potential users: archaeologists with expertise in rockart. The system allows them to explore the prehistoric art and its spatial context with highly realistic appearance. A set of dedicated interaction techniques was developed to facilitate collaborative visual analysis. A multi-display workspace supports the immediate comparison of geographically distributed artifacts. An expert review of the final demonstrator confirmed the potential for added value in rock-art research and the usability of our collaborative interaction techniques.

scholarly journals Medical data integration using HL7 standards for patient’s early identification

PLoS ONE ◽  
2021 ◽  
Vol 16 (12) ◽  
pp. e0262067
Author(s):  
Adi A. AlQudah ◽  
Mostafa Al-Emran ◽  
Khaled Shaalan
Keyword(s):  
Electronic Medical Records ◽  
Early Identification ◽  
Medical Records ◽  
Simulation Experiment ◽  
Waiting Times ◽  
Clinical Stage ◽  
Added Value ◽  
Significant Drop ◽  
Extensible Markup ◽  
Solution Design

Integration between information systems is critical, especially in the healthcare domain, since interoperability requirements are related to patients’ data confidentiality, safety, and satisfaction. The goal of this study is to propose a solution based on the integration between queue management solution (QMS) and the electronic medical records (EMR), using Health Level Seven (HL7) protocols and Extensible Markup Language (XML). The proposed solution facilitates the patient’s self-check-in within a healthcare organization in UAE. The solution aims to help in minimizing the waiting times within the outpatient department through early identification of patients who hold the Emirates national ID cards, i.e., whether an Emirati or expatriates. The integration components, solution design, and the custom-designed XML and HL7 messages were clarified in this paper. In addition, the study includes a simulation experiment through control and intervention weeks with 517 valid appointments. The experiment goal was to evaluate the patient’s total journey and each related clinical stage by comparing the “routine-based identification” with the “patient’s self-check-in” processes in case of booked appointments. As a key finding, the proposed solution is efficient and could reduce the “patient’s journey time” by more than 14 minutes and “time to identify” patients by 10 minutes. There was also a significant drop in the waiting time to triage and the time to finish the triage process. In conclusion, the proposed solution is considered innovative and can provide a positive added value for the patient’s whole journey.

Fast, Accurate, and Automated 3D Reconstruction Using a Depth Camera Mounted on an Industrial Robot

2021 ◽  
Author(s):  
Rishi Malhan ◽  
Rex Jomy Joseph ◽  
Prahar M. Bhatt ◽  
Brual Shah ◽  
Satyandra K. Gupta
Keyword(s):  
3D Reconstruction ◽  
Cycle Time ◽  
Industrial Robot ◽  
Low Cost ◽  
Point Clouds ◽  
Industrial Robots ◽  
Depth Camera ◽  
Reconstruction Process ◽  
Depth Cameras ◽  
Manufacturing Applications

Abstract 3D reconstruction technology is used in a wide variety of applications. Currently, automatically creating accurate pointclouds for large parts requires expensive hardware. We are interested in using low-cost depth cameras mounted on commonly available industrial robots to create accurate pointclouds for large parts automatically. Manufacturing applications require fast cycle times. Therefore, we are interested in speeding up the 3D reconstruction process. We present algorithmic advances in 3D reconstruction that achieve a sub-millimeter accuracy using a low-cost depth camera. Our system can be used to determine a pointcloud model of large and complex parts. Advances in camera calibration, cycle time reduction for pointcloud capturing, and uncertainty estimation are made in this work. We continuously capture point-clouds at an optimal camera location with respect to part distance during robot motion execution. The redundancy in pointclouds achieved by the moving camera significantly reduces errors in measurements without increasing cycle time. Our system produces sub-millimeter accuracy.

scholarly journals HBIM GENERATION: EXTENDING GEOMETRIC PRIMITIVES AND BIM MODELLING TOOLS FOR HERITAGE STRUCTURES AND COMPLEX VAULTED SYSTEMS

2019 ◽  
Vol XLII-2/W15 ◽  
pp. 139-148 ◽  
Author(s):  
F. Banfi
Keyword(s):  
Point Clouds ◽  
Added Value ◽  
Complex Objects ◽  
Digital Tool ◽  
Cloud Data ◽  
Wide Range ◽  
Building Information ◽  
Potential Applications ◽  
Building Life Cycle ◽  
Generative Modelling

<p><strong>Abstract.</strong> Today, the generation of smart models and digital archives able to accompany the management of architectural assets through the integration of advanced 3D survey techniques and Historic Building Information Modelling (HBIM) becomes an irremissible added value for the management of building life cycle. New international BIM standards are trying to give some guidelines to this new digital tool, which has demonstrated a wide range of potential applications in digital heritage, structural simulation, generative design, site management, restoration and daily maintenance activities. To manage a proper 3D reconstruction from point clouds, also known as Scan-to-BIM process, BIM experts typically use 3D application developed for the management of new buildings, which make it difficult to generate complex objects such as historic vaults and irregular and damaged walls. For these reasons, this study places the heart of its research in the integration of generative modelling in BIM application, proposing new Scan-to-BIM modelling requirements for complex vaulted systems and a novel add-in (a set of new functionalities for Autodesk Revit) able to improve the generation of complex elements from point cloud data. In particular, new modelling requirements such as novel grades of generation (GOG) and accuracy (GOA) are integrated within the Revit’s structure in order to improve the level of information connected to HBIM and consequently to support the decision making of restoration and rehabilitation projects of complex heritage structures.</p>

scholarly journals Road Rutting Measurement Using Mobile LiDAR Systems Point Cloud

2019 ◽  
Vol 8 (9) ◽  
pp. 404 ◽  
Author(s):  
Luis Gézero ◽  
Carlos Antunes
Keyword(s):  
Cross Section ◽  
Cross Sections ◽  
Point Clouds ◽  
Added Value ◽  
Time Saving ◽  
Pavement Distress ◽  
Vehicle Loading ◽  
Cloud Points ◽  
The Cross ◽  
Asset Inventory

Road rutting caused by vehicle loading in the wheel path is a major form of asphalt pavement distress. Hydroplaning and loss of skid resistance are directly related to high road rutting severity. Periodical measurements of rut depth are crucial to maintenance and rehabilitation planning. In this study, we explored the feasibility of using point clouds gathered by Mobile LiDAR systems to measure the rut depth. These point clouds that are collected along roads are usually used for other purposes, namely asset inventory or topographic survey. Taking advantage of available clouds to identify rutting severity in critical pavement areas can result in considerable economic and time saving and thus, added value, when compared with specific expensive rut measuring systems. Four different strategies of cloud points aggregation are presented to create the cross-section of points. Such strategies were established to improve the precision of individual sensor measurements. Despite the 5 mm precision of the used system, it was possible to estimate rut depth values that were slightly inferior. The rut depth values obtained from each cross-section strategy were compared with the manual field measured values. The cross-sections based on averaged cloud points sensor profile aggregation was revealed to be the most suitable strategy to measure rut depth. Despite the fact that the study was specifically conducted to measure rut depth, the evaluation results show that the methodology can also be useful for other mobile LiDAR point clouds cross-sections applications.

scholarly journals 3D SCANNING OF LIVE PIGS SYSTEM AND ITS APPLICATION IN BODY MEASUREMENTS

2017 ◽  
Vol XLII-2/W7 ◽  
pp. 211-217 ◽  
Author(s):  
H. Guo ◽  
K. Wang ◽  
W. Su ◽  
D. H. Zhu ◽  
W. L. Liu ◽  
...  
Keyword(s):  
Detection Method ◽  
Point Clouds ◽  
3D Scanning ◽  
Scanning System ◽  
Prototype System ◽  
Body Measurements ◽  
Important Indicator ◽  
Depth Cameras ◽  
3D Point Clouds ◽  
Common Coordinate System

The shape of a live pig is an important indicator of its health and value, whether for breeding or for carcass quality. This paper implements a prototype system for live single pig body surface 3d scanning based on two consumer depth cameras, utilizing the 3d point clouds data. These cameras are calibrated in advance to have a common coordinate system. The live 3D point clouds stream of moving single pig is obtained by two Xtion Pro Live sensors from different viewpoints simultaneously. A novel detection method is proposed and applied to automatically detect the frames containing pigs with the correct posture from the point clouds stream, according to the geometric characteristics of pig’s shape. The proposed method is incorporated in a hybrid scheme, that serves as the preprocessing step in a body measurements framework for pigs. Experimental results show the portability of our scanning system and effectiveness of our detection method. Furthermore, an updated this point cloud preprocessing software for livestock body measurements can be downloaded freely from <a href="https://github.com/LiveStockShapeAnalysis"target="_blank">https://github.com/LiveStockShapeAnalysis</a> to livestock industry, research community and can be used for monitoring livestock growth status.

scholarly journals Persistence Landscape based Topological Data Analysis for Personalized Arrhythmia Classification

2019 ◽  
Author(s):  
Yan Yan ◽  
Kamen Ivanov ◽  
Jian Cen ◽  
Qiu-Hua Liu ◽  
Lei Wang
Keyword(s):  
Data Analysis ◽  
High Performance ◽  
Topological Analysis ◽  
Point Clouds ◽  
Topological Data Analysis ◽  
Reconstruction Technique ◽  
Biomedical Data ◽  
Training Set ◽  
Set Size ◽  
Topological Data

Data can be illustrated in shapes, and the shapes could provide insight for data modeling and information extraction. Topological data analysis provides an alternative insight in biomedical data analysis and knowledge discovery with the algebra topology tools. In present work, we study the application of topological data analysis for personalized electrocardiographic signal classification toward arrhythmia analysis. Using phase space reconstruction technique, the signal samples are converted into point clouds for topological analysis facility. With topological techniques the persistence landscapes from the point clouds are extracted as features to perform the arrhythmia classification task. We find that the proposed method is robust to the training set size, with only a training set size of 20% percents, the normal heartbeat class are 100% recognized, ventricular beats for 97.13%, supra-ventricular beats for 94.27% and fusion beats for 94.27% within the corresponding experiments. The property of keeping high performance when using smaller training sample proves that the proposed method is especially applicable to personalized analysis. With the present study, we show that the topological data analysis technique could be a useful tool in biomedical signal analysis, and provide powerful ability in personalized analysis.

Sign in / Sign up

Export Citation Format

Share Document