scholarly journals Hole-Filling Algorithm with Spatio-Temporal Background Information for View Synthesis

2017 ◽  
Vol E100.D (9) ◽  
pp. 1994-2004
Author(s):  
Huu-Noi DOAN ◽  
Tien-Dat NGUYEN ◽  
Min-Cheol HONG
Keyword(s):  
View Synthesis ◽  
Background Information ◽  
Hole Filling ◽  
Spatio Temporal

Passthrough+

2020 ◽  
Vol 3 (1) ◽  
pp. 1-17
Author(s):  
Gaurav Chaurasia ◽  
Arthur Nieuwoudt ◽  
Alexandru-Eugen Ichim ◽  
Richard Szeliski ◽  
Alexander Sorkine-Hornung
Keyword(s):  
Real Time ◽  
Video Encoding ◽  
View Synthesis ◽  
Performance Constraints ◽  
Trade Offs ◽  
Stereoscopic View ◽  
Target User ◽  
Spatio Temporal ◽  
And Performance ◽  
3D Scene

We present an end-to-end system for real-time environment capture, 3D reconstruction, and stereoscopic view synthesis on a mobile VR headset. Our solution allows the user to use the cameras on their VR headset as their eyes to see and interact with the real world while still wearing their headset, a feature often referred to as Passthrough. The central challenge when building such a system is the choice and implementation of algorithms under the strict compute, power, and performance constraints imposed by the target user experience and mobile platform. A key contribution of this paper is a complete description of a corresponding system that performs temporally stable passthrough rendering at 72 Hz with only 200 mW power consumption on a mobile Snapdragon 835 platform. Our algorithmic contributions for enabling this performance include the computation of a coarse 3D scene proxy on the embedded video encoding hardware, followed by a depth densification and filtering step, and finally stereoscopic texturing and spatio-temporal up-sampling. We provide a detailed discussion and evaluation of the challenges we encountered, as well as algorithm and performance trade-offs in terms of compute and resulting passthrough quality.;AB@The described system is available to users as the Passthrough+ feature on Oculus Quest. We believe that by publishing the underlying system and methods, we provide valuable insights to the community on how to design and implement real-time environment sensing and rendering on heavily resource constrained hardware.

Disocclusion-type aware hole filling method for view synthesis

2021 ◽  
Author(s):  
Xiaodong Chen ◽  
Haitao Liang ◽  
Huaiyuan Xu ◽  
Siyu Ren ◽  
Huaiyu Cai ◽  
...  
Keyword(s):  
View Synthesis ◽  
Hole Filling ◽  
Filling Method

Adaptive hole filling for 3D warping-based virtual view synthesis

2013 ◽  
Author(s):  
Tatsuro Mori ◽  
Mehrdad Panahpour Tehrani ◽  
Toshiaki Fujii ◽  
Masayuki Tanimoto
Keyword(s):  
View Synthesis ◽  
Hole Filling ◽  
Virtual View Synthesis ◽  
Virtual View

scholarly journals Visual Features with Spatio-Temporal-Based Fusion Model for Cross-Dataset Vehicle Re-Identification

Electronics ◽  
2020 ◽  
Vol 9 (7) ◽  
pp. 1083 ◽  
Author(s):  
Zakria ◽  
Jianhua Deng ◽  
Jingye Cai ◽  
Muhammad Umar Aftab ◽  
Muhammad Saddam Khokhar ◽  
...  
Keyword(s):  
Neural Network ◽  
Temporal Patterns ◽  
Extensive Study ◽  
Background Information ◽  
Training Dataset ◽  
Visual Features ◽  
Surveillance Network ◽  
Significant Difference ◽  
Spatio Temporal ◽  
The Impact

Vehicle re-identification (Re-Id) is the key module in an intelligent transportation system (ITS). Due to its versatile applicability in metropolitan cities, this task has received increasing attention these days. It aims to identify whether the specific vehicle has already appeared over the surveillance network or not. Mostly, the vehicle Re-Id method are evaluated on a single dataset, in which training and testing of the model is performed on the same dataset. However in practice, this negatively effects model generalization ability due to biased datasets along with the significant difference between training and testing data; hence, the model becomes weak in a practical environment. To demonstrate this issue, we have empirically shown that the current vehicle Re-Id datasets are usually strongly biased. In this regard, we also conduct an extensive study on the cross and the same dataset to examine the impact on the performance of the vehicle Re-Id system, considering existing methods. To address the problem, in this paper, we have proposed an approach with augmentation of the training dataset to reduce the influence of pose, angle, camera color response, and background information in vehicle images; whereas, spatio-temporal patterns of unlabelled target datasets are learned by transferring siamese neural network classifiers trained on a source-labelled dataset. We finally calculate the composite similarity score of spatio-temporal patterns with siamese neural-network-based classifier visual features. Extensive experiments on multiple datasets are examined and results suggest that the proposed approach has the ability to generalize adequately.

Sign in / Sign up

Export Citation Format

Share Document