Virtual Reality Providing Ground Truth for Machine Learning and Programming by Demonstration

2012 ◽  
Author(s):  
Jürgen Roßmann ◽  
Christian Schlette ◽  
Nils Wantia
Keyword(s):  
Virtual Reality ◽  
Space Station ◽  
Ground Truth ◽  
Cognitive System ◽  
Automated System ◽  
Ground Truth Data ◽  
Free Data ◽  
Funded Project ◽  
Manual Tasks ◽  
Bounding Boxes

In this contribution, it is demonstrated how the development, parameterization and evaluation of an intelligent cognitive system can be supported by means of Virtual Reality (VR). Based on new approaches in object and action recognition, the consortium in the EU funded project IntellAct is developing such an automated system for the intelligent understanding of manual tasks and robot-based manipulations. In this context, the VR setup of the Institute for Man-Machine Interaction (MMI) is able to deliver ideal data, such as detailed positions and joint data of objects and kinematics involved. In addition, it allows for the synthesis of distinct and error-free data, such as collisions, bounding boxes and contact events. This way, Virtual Reality serves as a reference and source for “ground truth” data for designing, training and benchmarking the intelligent cognitive system during its development and parameterization. Furthermore this approach enables descriptive and secure ways for the visualization of results in the targeted environments, such as the highly automated laboratories on-board the International Space Station (see Fig. 1).

Enhancing unsupervised video-based vehicle tracking and modeling for traffic data collection

2020 ◽  
Vol 47 (8) ◽  
pp. 982-997
Author(s):  
Mohamed H. Zaki ◽  
Tarek Sayed ◽  
Moataz Billeh
Keyword(s):  
Data Collection ◽  
Video Analysis ◽  
Three Dimensional ◽  
Ground Truth ◽  
Traffic Data ◽  
Ground Truth Data ◽  
The Road ◽  
Traffic Data Collection ◽  
Bounding Boxes ◽  
Automated Video Analysis

Video-based traffic analysis is a leading technology for streamlining transportation data collection. With traffic records from video cameras, unsupervised automated video analysis can detect various vehicle measures such as vehicle spatial coordinates and subsequently lane positions, speed, and other dynamic measures without the need of any physical interconnections to the road infrastructure. This paper contributes to the unsupervised automated video analysis by addressing two main shortcomings of the approach. The first objective is to alleviate tracking problems of over-segmentation and over-grouping by integrating region-based detection with feature-based tracking. This information, when combined with spatiotemporal constraints of grouping, can reduce the effects of these problems. This fusion approach offers a superior decision procedure for grouping objects and discriminating between trajectories of objects. The second objective is to model three-dimensional bounding boxes for the vehicles, leading to a better estimate of their geometry and consequently accurate measures of their position and travel information. This improvement leads to more precise measurement of traffic parameters such as average speed, gap time, and headway. The paper describes the various steps of the proposed improvements. It evaluates the effectiveness of the refinement process on data collected from traffic cameras in three different locations in Canada and validates the results with ground truth data. It illustrates the effectiveness of the improved unsupervised automated video analysis with a case study on 10 h of traffic data collection such as volume and headway measurements.

scholarly journals 3D Vehicle Trajectory Extraction Using DCNN in an Overlapping Multi-Camera Crossroad Scene

Sensors ◽  
2021 ◽  
Vol 21 (23) ◽  
pp. 7879
Author(s):  
Jinyeong Heo ◽  
Yongjin (James) Kwon
Keyword(s):  
Heavy Traffic ◽  
Ground Plane ◽  
Ground Truth ◽  
Autonomous Driving ◽  
Linear Interpolation ◽  
Single Camera ◽  
Ground Truth Data ◽  
Wide Range ◽  
Vehicle Trajectory ◽  
Bounding Boxes

The 3D vehicle trajectory in complex traffic conditions such as crossroads and heavy traffic is practically very useful in autonomous driving. In order to accurately extract the 3D vehicle trajectory from a perspective camera in a crossroad where the vehicle has an angular range of 360 degrees, problems such as the narrow visual angle in single-camera scene, vehicle occlusion under conditions of low camera perspective, and lack of vehicle physical information must be solved. In this paper, we propose a method for estimating the 3D bounding boxes of vehicles and extracting trajectories using a deep convolutional neural network (DCNN) in an overlapping multi-camera crossroad scene. First, traffic data were collected using overlapping multi-cameras to obtain a wide range of trajectories around the crossroad. Then, 3D bounding boxes of vehicles were estimated and tracked in each single-camera scene through DCNN models (YOLOv4, multi-branch CNN) combined with camera calibration. Using the abovementioned information, the 3D vehicle trajectory could be extracted on the ground plane of the crossroad by calculating results obtained from the overlapping multi-camera with a homography matrix. Finally, in experiments, the errors of extracted trajectories were corrected through a simple linear interpolation and regression, and the accuracy of the proposed method was verified by calculating the difference with ground-truth data. Compared with other previously reported methods, our approach is shown to be more accurate and more practical.

CLUBS: An RGB-D dataset with cluttered box scenes containing household objects

2019 ◽  
Vol 38 (14) ◽  
pp. 1538-1548
Author(s):  
Tonci Novkovic ◽  
Fadri Furrer ◽  
Marko Panjek ◽  
Margarita Grinvald ◽  
Roland Siegwart ◽  
...  
Keyword(s):  
Ground Truth ◽  
Calibration Data ◽  
Ground Truth Data ◽  
Depth Images ◽  
Difficult Time ◽  
Rgb Images ◽  
Bounding Boxes ◽  
Realistic Scenes ◽  
Intrinsic And Extrinsic Parameters ◽  
Realistic Data

With the progress of machine learning, the demand for realistic data with high-quality annotations has been thriving. In order to generalize well, considerable amounts of data are required, especially realistic ground-truth data, for tasks such as object detection and scene segmentation. Such data can be difficult, time-consuming, and expensive to collect. This article presents a dataset of household objects and box scenes commonly found in warehouse environments. The dataset was obtained using a robotic setup with four different cameras. It contains reconstructed objects and scenes, as well as raw RGB and depth images, camera poses, pixel-wise labels of objects directly in the RGB images, and 3D bounding boxes with poses in the world frame. Furthermore, raw calibration data are provided, together with the intrinsic and extrinsic parameters for all the sensors. By providing object labels as pixel-wise masks, 3D, and 2D object bounding boxes, this dataset is useful for both object recognition and instance segmentation. The realistic scenes provided will serve for learning-based algorithms applied to scenarios where boxes of objects are often found, such as in the logistics sector. Both the dataset and the tools for data processing are published and available online.

scholarly journals Assessing Wildfire Burn Severity and Its Relationship with Environmental Factors: A Case Study in Interior Alaska Boreal Forest

2021 ◽  
Vol 13 (10) ◽  
pp. 1966
Author(s):  
Christopher W Smith ◽  
Santosh K Panda ◽  
Uma S Bhatt ◽  
Franz J Meyer ◽  
Anushree Badola ◽  
...  
Keyword(s):  
Boreal Forest ◽  
Ground Truth ◽  
Burn Severity ◽  
Classification Methods ◽  
Spectral Indices ◽  
Ground Truth Data ◽  
Burn Scar ◽  
Interior Alaska ◽  
Remote Sensing Methods ◽  
The Relationship

In recent years, there have been rapid improvements in both remote sensing methods and satellite image availability that have the potential to massively improve burn severity assessments of the Alaskan boreal forest. In this study, we utilized recent pre- and post-fire Sentinel-2 satellite imagery of the 2019 Nugget Creek and Shovel Creek burn scars located in Interior Alaska to both assess burn severity across the burn scars and test the effectiveness of several remote sensing methods for generating accurate map products: Normalized Difference Vegetation Index (NDVI), Normalized Burn Ratio (NBR), and Random Forest (RF) and Support Vector Machine (SVM) supervised classification. We used 52 Composite Burn Index (CBI) plots from the Shovel Creek burn scar and 28 from the Nugget Creek burn scar for training classifiers and product validation. For the Shovel Creek burn scar, the RF and SVM machine learning (ML) classification methods outperformed the traditional spectral indices that use linear regression to separate burn severity classes (RF and SVM accuracy, 83.33%, versus NBR accuracy, 73.08%). However, for the Nugget Creek burn scar, the NDVI product (accuracy: 96%) outperformed the other indices and ML classifiers. In this study, we demonstrated that when sufficient ground truth data is available, the ML classifiers can be very effective for reliable mapping of burn severity in the Alaskan boreal forest. Since the performance of ML classifiers are dependent on the quantity of ground truth data, when sufficient ground truth data is available, the ML classification methods would be better at assessing burn severity, whereas with limited ground truth data the traditional spectral indices would be better suited. We also looked at the relationship between burn severity, fuel type, and topography (aspect and slope) and found that the relationship is site-dependent.

scholarly journals Automatic Evaluation of Wheat Resistance to Fusarium Head Blight Using Dual Mask-RCNN Deep Learning Frameworks in Computer Vision

2020 ◽  
Vol 13 (1) ◽  
pp. 26
Author(s):  
Wen-Hao Su ◽  
Jiajing Zhang ◽  
Ce Yang ◽  
Rae Page ◽  
Tamas Szinyei ◽  
...  
Keyword(s):  
Fusarium Head Blight ◽  
Ground Truth ◽  
Wheat Breeding ◽  
Head Blight ◽  
Detection Rates ◽  
Ground Truth Data ◽  
Resistant Cultivars ◽  
Feature Pyramid ◽  
Rater Error ◽  
Wheat Lines

In many regions of the world, wheat is vulnerable to severe yield and quality losses from the fungus disease of Fusarium head blight (FHB). The development of resistant cultivars is one means of ameliorating the devastating effects of this disease, but the breeding process requires the evaluation of hundreds of lines each year for reaction to the disease. These field evaluations are laborious, expensive, time-consuming, and are prone to rater error. A phenotyping cart that can quickly capture images of the spikes of wheat lines and their level of FHB infection would greatly benefit wheat breeding programs. In this study, mask region convolutional neural network (Mask-RCNN) allowed for reliable identification of the symptom location and the disease severity of wheat spikes. Within a wheat line planted in the field, color images of individual wheat spikes and their corresponding diseased areas were labeled and segmented into sub-images. Images with annotated spikes and sub-images of individual spikes with labeled diseased areas were used as ground truth data to train Mask-RCNN models for automatic image segmentation of wheat spikes and FHB diseased areas, respectively. The feature pyramid network (FPN) based on ResNet-101 network was used as the backbone of Mask-RCNN for constructing the feature pyramid and extracting features. After generating mask images of wheat spikes from full-size images, Mask-RCNN was performed to predict diseased areas on each individual spike. This protocol enabled the rapid recognition of wheat spikes and diseased areas with the detection rates of 77.76% and 98.81%, respectively. The prediction accuracy of 77.19% was achieved by calculating the ratio of the wheat FHB severity value of prediction over ground truth. This study demonstrates the feasibility of rapidly determining levels of FHB in wheat spikes, which will greatly facilitate the breeding of resistant cultivars.

scholarly journals RobotP: A Benchmark Dataset for 6D Object Pose Estimation

Sensors ◽  
2021 ◽  
Vol 21 (4) ◽  
pp. 1299
Author(s):  
Honglin Yuan ◽  
Tim Hoogenkamp ◽  
Remco C. Veltkamp
Keyword(s):  
Pose Estimation ◽  
Ground Truth ◽  
3D Models ◽  
Depth Image ◽  
Great Success ◽  
Estimation Algorithms ◽  
Depth Images ◽  
Object Pose Estimation ◽  
Image Pairs ◽  
Bounding Boxes

Deep learning has achieved great success on robotic vision tasks. However, when compared with other vision-based tasks, it is difficult to collect a representative and sufficiently large training set for six-dimensional (6D) object pose estimation, due to the inherent difficulty of data collection. In this paper, we propose the RobotP dataset consisting of commonly used objects for benchmarking in 6D object pose estimation. To create the dataset, we apply a 3D reconstruction pipeline to produce high-quality depth images, ground truth poses, and 3D models for well-selected objects. Subsequently, based on the generated data, we produce object segmentation masks and two-dimensional (2D) bounding boxes automatically. To further enrich the data, we synthesize a large number of photo-realistic color-and-depth image pairs with ground truth 6D poses. Our dataset is freely distributed to research groups by the Shape Retrieval Challenge benchmark on 6D pose estimation. Based on our benchmark, different learning-based approaches are trained and tested by the unified dataset. The evaluation results indicate that there is considerable room for improvement in 6D object pose estimation, particularly for objects with dark colors, and photo-realistic images are helpful in increasing the performance of pose estimation algorithms.

scholarly journals Classification of Cattle Behaviours Using Neck-Mounted Accelerometer-Equipped Collars and Convolutional Neural Networks

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 4050
Author(s):  
Dejan Pavlovic ◽  
Christopher Davison ◽  
Andrew Hamilton ◽  
Oskar Marko ◽  
Robert Atkinson ◽  
...  
Keyword(s):  
Neural Network ◽  
Model Performance ◽  
Ground Truth ◽  
Practical Implementation ◽  
Ground Truth Data ◽  
Battery Lifetime ◽  
Implementation Challenges ◽  
Memory Footprint ◽  
Commercial Farms ◽  
Using Data

Monitoring cattle behaviour is core to the early detection of health and welfare issues and to optimise the fertility of large herds. Accelerometer-based sensor systems that provide activity profiles are now used extensively on commercial farms and have evolved to identify behaviours such as the time spent ruminating and eating at an individual animal level. Acquiring this information at scale is central to informing on-farm management decisions. The paper presents the development of a Convolutional Neural Network (CNN) that classifies cattle behavioural states (`rumination’, `eating’ and `other’) using data generated from neck-mounted accelerometer collars. During three farm trials in the United Kingdom (Easter Howgate Farm, Edinburgh, UK), 18 steers were monitored to provide raw acceleration measurements, with ground truth data provided by muzzle-mounted pressure sensor halters. A range of neural network architectures are explored and rigorous hyper-parameter searches are performed to optimise the network. The computational complexity and memory footprint of CNN models are not readily compatible with deployment on low-power processors which are both memory and energy constrained. Thus, progressive reductions of the CNN were executed with minimal loss of performance in order to address the practical implementation challenges, defining the trade-off between model performance versus computation complexity and memory footprint to permit deployment on micro-controller architectures. The proposed methodology achieves a compression of 14.30 compared to the unpruned architecture but is nevertheless able to accurately classify cattle behaviours with an overall F1 score of 0.82 for both FP32 and FP16 precision while achieving a reasonable battery lifetime in excess of 5.7 years.

A machine learning approach to estimate the strain energy absorption in expanded polystyrene foams

2021 ◽  
pp. 0021955X2110210
Author(s):  
Alejandro E Rodríguez-Sánchez ◽  
Héctor Plascencia-Mora
Keyword(s):  
Neural Network ◽  
Energy Absorption ◽  
Mechanical Energy ◽  
Compressive Loading ◽  
Ground Truth ◽  
Expanded Polystyrene ◽  
Polystyrene Foam ◽  
Stress Strain ◽  
Ground Truth Data ◽  
Expanded Polystyrene Foam

Traditional modeling of mechanical energy absorption due to compressive loadings in expanded polystyrene foams involves mathematical descriptions that are derived from stress/strain continuum mechanics models. Nevertheless, most of those models are either constrained using the strain as the only variable to work at large deformation regimes and usually neglect important parameters for energy absorption properties such as the material density or the rate of the applying load. This work presents a neural-network-based approach that produces models that are capable to map the compressive stress response and energy absorption parameters of an expanded polystyrene foam by considering its deformation, compressive loading rates, and different densities. The models are trained with ground-truth data obtained in compressive tests. Two methods to select neural network architectures are also presented, one of which is based on a Design of Experiments strategy. The results show that it is possible to obtain a single artificial neural networks model that can abstract stress and energy absorption solution spaces for the conditions studied in the material. Additionally, such a model is compared with a phenomenological model, and the results show than the neural network model outperforms it in terms of prediction capabilities, since errors around 2% of experimental data were obtained. In this sense, it is demonstrated that by following the presented approach is possible to obtain a model capable to reproduce compressive polystyrene foam stress/strain data, and consequently, to simulate its energy absorption parameters.

scholarly journals Multi-Temporal Arable Land Monitoring in Arid Region of Northwest China Using a New Extraction Index

2021 ◽  
Vol 13 (9) ◽  
pp. 5274
Author(s):  
Xinyang Yu ◽  
Younggu Her ◽  
Xicun Zhu ◽  
Changhe Lu ◽  
Xuefei Li
Keyword(s):  
Arable Land ◽  
Ground Truth ◽  
Northwest China ◽  
Hexi Corridor ◽  
Ground Truth Data ◽  
Land Protection ◽  
Promising Tool ◽  
Study Results ◽  
Multi Temporal ◽  
The Mean

Development of a high-accuracy method to extract arable land using effective data sources is crucial to detect and monitor arable land dynamics, servicing land protection and sustainable development. In this study, a new arable land extraction index (ALEI) based on spectral analysis was proposed, examined by ground truth data, and then applied to the Hexi Corridor in northwest China. The arable land and its change patterns during 1990–2020 were extracted and identified using 40 Landsat TM/OLI images acquired in 1990, 2000, 2010, and 2020. The results demonstrated that the proposed method can distinguish arable land areas accurately, with the User’s (Producer’s) accuracy and overall accuracy (kappa coefficient) exceeding 0.90 (0.88) and 0.89 (0.87), respectively. The mean relative error calculated using field survey data obtained in 2012 and 2020 was 0.169 and 0.191, respectively, indicating the feasibility of the ALEI method in arable land extracting. The study found that arable land area in the Hexi Corridor was 13217.58 km2 in 2020, significantly increased by 25.33% compared to that in 1990. At 10-year intervals, the arable land experienced different change patterns. The study results indicate that ALEI index is a promising tool used to effectively extract arable land in the arid area.

scholarly journals Detection of heat pumps from smart meter and open data

2020 ◽  
Vol 3 (S1) ◽  
Author(s):  
Andreas Weigert ◽  
Konstantin Hopf ◽  
Nicolai Weinig ◽  
Thorsten Staake
Keyword(s):  
Heat Pump ◽  
Open Data ◽  
Ground Truth ◽  
Heat Pumps ◽  
Geographical Information ◽  
Weather Data ◽  
Smart Meters ◽  
Ground Truth Data ◽  
Thermal Reservoir ◽  
Grid Operators

Abstract Heat pumps embody solutions that heat or cool buildings effectively and sustainably, with zero emissions at the place of installation. As they pose significant load on the power grid, knowledge on their existence is crucial for grid operators, e.g., to forecast load and to plan grid operation. Further details, like the thermal reservoir (ground or air source) or the age of a heat pump installation renders energy-related services possible that utility companies can offer in the future (e.g., detecting wrongly calibrated installations, household energy efficiency checks). This study investigates the prediction of heat pump installations, their thermal reservoir and age. For this, we obtained a dataset with 397 households in Switzerland, all equipped with smart meters, collected ground truth data on installed heat pumps and enriched this data with weather data and geographical information. Our investigation replicates the state of the art in the area of heat pump detection and goes beyond it, as we obtain three major findings: First, machine learning can detect the existence of heat pumps with an AUC performance metric of 0.82, their heat reservoir with an AUC of 0.86, and their age with an AUC of 0.73. Second, heat pump existence can be better detected using data during the heating period than during summer. Third the number of training samples to detect the existence of heat pumps must not be necessarily large in terms of the number of training instances and observation period.

Sign in / Sign up

Export Citation Format

Share Document