Ground truth and benchmarks for performance evaluation

2003 ◽  
Author(s):  
Ayako Takeuchi ◽  
Michael Shneier ◽  
Tsai Hong Hong ◽  
Tommy Chang ◽  
Christopher Scrapper ◽  
...  
Keyword(s):  
Performance Evaluation ◽  
Ground Truth

scholarly journals Performance Evaluation of Ground AR Anchor with WebXR Device API

2021 ◽  
Vol 11 (17) ◽  
pp. 7877
Author(s):  
Daehyeon Lee ◽  
Woosung Shim ◽  
Munyong Lee ◽  
Seunghyun Lee ◽  
Kye-Dong Jung ◽  
...  
Keyword(s):  
Performance Evaluation ◽  
Mixed Reality ◽  
Ground Truth ◽  
Application Programming Interface ◽  
Absolute Error ◽  
Frame Rate ◽  
Difference Image ◽  
Match Rate ◽  
The Difference ◽  
The Web

Recently, the development of 3D graphics technology has led to various technologies being combined with reality, where a new reality is defined or studied; they are typically named by combining the name of the technology with “reality”. Representative “reality” includes Augmented Reality, Virtual Reality, Mixed Reality, and eXtended Reality (XR). In particular, research on XR in the web environment is actively being conducted. The Web eXtended Reality Device Application Programming Interface (WebXR Device API), released in 2018, allows instant deployment of XR services to any XR platform requiring only an active web browser. However, the currently released tentative version has poor stability. Therefore, in this study, the performance evaluation of WebXR Device API is performed using three experiments. A camera trajectory experiment is analyzed using ground truth, we checked the standard deviation between the ground truth and WebXR for the X, Y, and Z axes. The difference image experiment is conducted for the front, left, and right directions, which resulted in a visible difference image for each image of ground truth and WebXR, small mean absolute error, and high match rate. In the experiment for measuring the 3D rendering speed, a frame rate similar to that of real-time is obtained.

Performance Evaluation of Real-Time Kinematic GPS at Arctic Latitudes

2009 ◽  
Author(s):  
Alessio Salerno ◽  
Tom Lamarche ◽  
Erick Dupuis
Keyword(s):  
Performance Evaluation ◽  
Real Time ◽  
Global Positioning System ◽  
Ground Truth ◽  
Research Network ◽  
Real Time Kinematic ◽  
Global Positioning ◽  
Challenging Environment ◽  
Analogue Research ◽  
Rtk Gps

A real-time kinematic (RTK) global positioning system (GPS) has been identified for potentially being used as a ground-truth sensor for testing robotic rovers for planetary exploration. A series of environmental tests needs to be performed in order to validate the performance of the sensor at hand before being used as a ground-truth system. This paper focuses on the performance evaluation of the RTK GPS at Axel Heiberg Island Canadian Space Agency’s Analogue Research Network (CARN) site. This is one of the officially recognized terrestrial analogues, that is places on Earth that approximate the geological, environmental and putative biological conditions on Mars and other planetary bodies (Hipkin et al.). The challenge lies in the use of the equipment at Arctic latitudes. The results show that the system performed according to specifications even in this challenging environment.

Some Fuzzy Tools for Evaluation of Computer Vision Algorithms

2018 ◽  
Vol 8 (1) ◽  
pp. 1-14
Author(s):  
Andrey Osipov
Keyword(s):  
Computer Vision ◽  
Performance Evaluation ◽  
Face Recognition ◽  
Set Theory ◽  
Fuzzy Set Theory ◽  
Evaluation Method ◽  
Similarity Measures ◽  
Ground Truth ◽  
Fuzzy Similarity ◽  
Segmentation Quality

In this article, some issues related to the performance evaluation of computer vision algorithms within the version of direct empirical supervised evaluation method developed at SRISA RAS are considered. This approach partly relies on the elements defined by using the fuzzy set theory, in particular, fuzzy similarity measures and fuzzy reference ground truth images. Some known measures of segmentation quality are considered and their extensions, representing the fuzzy similarity measures, are offered. As an example, the author considers an application of fuzzy ground truth images and fuzzy similarity measures, including some newly introduced ones, to the evaluation of face recognition algorithms.

scholarly journals A Novel Experiment-Free Site-Specific TDoA Localization Performance-Evaluation Approach

Sensors ◽  
2020 ◽  
Vol 20 (4) ◽  
pp. 1035
Author(s):  
Xiaokang Ye ◽  
José Rodríguez-Piñeiro ◽  
Yuan Liu ◽  
Xuefeng Yin ◽  
Antonio Pérez Yuste
Keyword(s):  
Performance Evaluation ◽  
Ground Truth ◽  
Integrated Approach ◽  
Impulse Responses ◽  
Evaluation Approach ◽  
Localization Errors ◽  
Localization Performance ◽  
Transmitted Signal ◽  
Evaluation Approaches ◽  
Localization Systems

Time difference of arrival (TDoA) technology is widely utilized for source localization, which stimulates many studies on performance-evaluation approaches for TDoA localization systems. Some approaches using simulations are designed merely for a simple Line-of-Sight (LoS) scenario while some other ones using experiments show high cost and inefficiency. This paper proposes an integrated approach to evaluate a TDoA localization system in an area with a complicated environment. Radio propagation graph is applied through a simulation to obtain channel impulse responses (CIRs) between a source to be located and the TDoA sensors for the area. Realistic signals received by the sensors in baseband are emulated combining the source transmitted signal and the CIRs. A hardware unit takes charge of sending the radio emulated received signals to the system under test, which is consistent with real experimental measurements. Statistical analysis of the system is allowed based on localization errors obtained comparing the system’s estimates with the ground truth of the source location. Verified results for LoS and non-LoS scenarios with variable transmitted signal bandwidths and signal-to-noise ratios, as well as for three variations of the sensor locations in an automobile circuit, show the usability of the proposed experiment-free performance-evaluation approach.

Performance Evaluation Based on Information Theory for Multi-Target Tracking Algorithm

2013 ◽  
Vol 694-697 ◽  
pp. 2341-2344
Author(s):  
Shu Rong Tian ◽  
Xiao Shu Sun ◽  
Dan Liu
Keyword(s):  
Information Theory ◽  
Performance Evaluation ◽  
Target Tracking ◽  
Ground Truth ◽  
Tracking Algorithm ◽  
Tracking Performance ◽  
Actual State ◽  
Target Number ◽  
Multi Target Tracking

This paper is concerned with the performance evaluation of algorithm of multi-target and target types tracking. Performance evaluation is based on information theory, Kullback-Leibler measure is used to discriminate information provided by algorithm. Through simulations, algorithm of multi-target tracking was evaluated in term of information (localization, classification, and target number components) the algorithm provide about the actual state of ground truth.

Sign in / Sign up

Export Citation Format

Share Document