Augmented reality through real-time tracking of video sequences using a panoramic view

2004 ◽  
Author(s):  
C. Dehais ◽  
M. Douze ◽  
G. Morin ◽  
V. Charvillat
Keyword(s):  
Augmented Reality ◽  
Real Time ◽  
Video Sequences ◽  
Panoramic View ◽  
Real Time Tracking

scholarly journals An integrated augmented reality surgical navigation platform using multi-modality imaging for guidance

PLoS ONE ◽  
2021 ◽  
Vol 16 (4) ◽  
pp. e0250558
Author(s):  
Harley H. L. Chan ◽  
Stephan K. Haerle ◽  
Michael J. Daly ◽  
Jinzi Zheng ◽  
Lauren Philp ◽  
...  
Keyword(s):  
Augmented Reality ◽  
Real Time ◽  
Navigation System ◽  
Surgical Navigation ◽  
Multimodality Imaging ◽  
Anatomical Structures ◽  
Image Guided ◽  
Surgical Navigation System ◽  
Virtual Images ◽  
Real Time Tracking

An integrated augmented reality (AR) surgical navigation system that potentially improves intra-operative visualization of concealed anatomical structures. Integration of real-time tracking technology with a laser pico-projector allows the surgical surface to be augmented by projecting virtual images of lesions and critical structures created by multimodality imaging. We aim to quantitatively and qualitatively evaluate the performance of a prototype interactive AR surgical navigation system through a series of pre-clinical studies. Four pre-clinical animal studies using xenograft mouse models were conducted to investigate system performance. A combination of CT, PET, SPECT, and MRI images were used to augment the mouse body during image-guided procedures to assess feasibility. A phantom with machined features was employed to quantitatively estimate the system accuracy. All the image-guided procedures were successfully performed. The tracked pico-projector correctly and reliably depicted virtual images on the animal body, highlighting the location of tumour and anatomical structures. The phantom study demonstrates the system was accurate to 0.55 ± 0.33mm. This paper presents a prototype real-time tracking AR surgical navigation system that improves visualization of underlying critical structures by overlaying virtual images onto the surgical site. This proof-of-concept pre-clinical study demonstrated both the clinical applicability and high precision of the system which was noted to be accurate to <1mm.

scholarly journals MS-Faster R-CNN: Multi-Stream Backbone for Improved Faster R-CNN Object Detection and Aerial Tracking from UAV Images

2021 ◽  
Vol 13 (9) ◽  
pp. 1670
Author(s):  
Danilo Avola ◽  
Luigi Cinque ◽  
Anxhelo Diko ◽  
Alessio Fagioli ◽  
Gian Luca Foresti ◽  
...  
Keyword(s):  
Image Analysis ◽  
Object Detection ◽  
Real Time ◽  
Video Sequences ◽  
Multi Scale ◽  
Video Frames ◽  
Common Strategy ◽  
Real Time Tracking ◽  
Uav Images ◽  
Sort Algorithm

Tracking objects across multiple video frames is a challenging task due to several difficult issues such as occlusions, background clutter, lighting as well as object and camera view-point variations, which directly affect the object detection. These aspects are even more emphasized when analyzing unmanned aerial vehicles (UAV) based images, where the vehicle movement can also impact the image quality. A common strategy employed to address these issues is to analyze the input images at different scales to obtain as much information as possible to correctly detect and track the objects across video sequences. Following this rationale, in this paper, we introduce a simple yet effective novel multi-stream (MS) architecture, where different kernel sizes are applied to each stream to simulate a multi-scale image analysis. The proposed architecture is then used as backbone for the well-known Faster-R-CNN pipeline, defining a MS-Faster R-CNN object detector that consistently detects objects in video sequences. Subsequently, this detector is jointly used with the Simple Online and Real-time Tracking with a Deep Association Metric (Deep SORT) algorithm to achieve real-time tracking capabilities on UAV images. To assess the presented architecture, extensive experiments were performed on the UMCD, UAVDT, UAV20L, and UAV123 datasets. The presented pipeline achieved state-of-the-art performance, confirming that the proposed multi-stream method can correctly emulate the robust multi-scale image analysis paradigm.

Sign in / Sign up

Export Citation Format

Share Document