scholarly journals Fast and Accurate Light Field View Synthesis by Optimizing Input View Selection

Micromachines ◽  
2021 ◽  
Vol 12 (5) ◽  
pp. 557
Author(s):  
Xingzheng Wang ◽  
Yongqiang Zan ◽  
Senlin You ◽  
Yuanlong Deng ◽  
Lihua Li
Keyword(s):  
Spatial Resolution ◽  
Light Field ◽  
Angular Resolution ◽  
Super Resolution ◽  
Depth Estimation ◽  
View Synthesis ◽  
The Novel ◽  
Arbitrary Location ◽  
Field View

There is a trade-off between spatial resolution and angular resolution limits in light field applications; various targeted algorithms have been proposed to enhance angular resolution while ensuring high spatial resolution simultaneously, which is also called view synthesis. Among them, depth estimation-based methods can use only four corner views to reconstruct a novel view at an arbitrary location. However, depth estimation is a time-consuming process, and the quality of the reconstructed novel view is not only related to the number of the input views, but also the location of the input views. In this paper, we explore the relationship between different input view selections with the angular super-resolution reconstruction results. Different numbers and positions of input views are selected to compare the speed of super-resolution reconstruction and the quality of novel views. Experimental results show that the speed of the algorithm decreases with the increase of the input views for each novel view, and the quality of the novel view decreases with the increase of the distance from the input views. After comparison using two input views in the same line to reconstruct the novel views between them, fast and accurate light field view synthesis is achieved.

scholarly journals Learning Light Field Angular Super-Resolution via a Geometry-Aware Network

2020 ◽  
Vol 34 (07) ◽  
pp. 11141-11148 ◽  
Author(s):  
Jing Jin ◽  
Junhui Hou ◽  
Hui Yuan ◽  
Sam Kwong
Keyword(s):  
Light Field ◽  
Angular Resolution ◽  
State Of The Art ◽  
Super Resolution ◽  
Depth Estimation ◽  
High Angular Resolution ◽  
Intrinsic Geometry ◽  
Physically Based ◽  
Small Baseline ◽  
Significant Superiority

The acquisition of light field images with high angular resolution is costly. Although many methods have been proposed to improve the angular resolution of a sparsely-sampled light field, they always focus on the light field with a small baseline, which is captured by a consumer light field camera. By making full use of the intrinsic geometry information of light fields, in this paper we propose an end-to-end learning-based approach aiming at angularly super-resolving a sparsely-sampled light field with a large baseline. Our model consists of two learnable modules and a physically-based module. Specifically, it includes a depth estimation module for explicitly modeling the scene geometry, a physically-based warping for novel views synthesis, and a light field blending module specifically designed for light field reconstruction. Moreover, we introduce a novel loss function to promote the preservation of the light field parallax structure. Experimental results over various light field datasets including large baseline light field images demonstrate the significant superiority of our method when compared with state-of-the-art ones, i.e., our method improves the PSNR of the second best method up to 2 dB in average, while saves the execution time 48×. In addition, our method preserves the light field parallax structure better.

scholarly journals How depth estimation in light fields can benefit from super-resolution?

2018 ◽  
Vol 15 (1) ◽  
pp. 172988141774844 ◽  
Author(s):  
Mandan Zhao ◽  
Gaochang Wu ◽  
Yebin Liu ◽  
Xiangyang Hao
Keyword(s):  
High Resolution ◽  
Light Field ◽  
Three Dimensional ◽  
Super Resolution ◽  
Depth Estimation ◽  
Light Fields ◽  
Trade Off ◽  
Depth Maps ◽  
Field Imaging

With the development of consumer light field cameras, the light field imaging has become an extensively used method for capturing the three-dimensional appearance of a scene. The depth estimation often requires a dense sampled light field in the angular domain or a high resolution in the spatial domain. However, there is an inherent trade-off between the angular and spatial resolutions of the light field. Recently, some studies for super-resolving the trade-off light field have been introduced. Rather than the conventional approaches that optimize the depth maps, these approaches focus on maximizing the quality of the super-resolved light field. In this article, we investigate how the depth estimation can benefit from these super-resolution methods. Specifically, we compare the qualities of the estimated depth using (a) the original sparse sampled light fields and the reconstructed dense sampled light fields, and (b) the original low-resolution light fields and the high-resolution light fields. Experiment results evaluate the enhanced depth maps using different super-resolution approaches.

A Fast View Synthesis Implementation Method for Light Field Applications

2021 ◽  
Vol 17 (4) ◽  
pp. 1-20
Author(s):  
Wei Gao ◽  
Linjie Zhou ◽  
Lvfang Tao
Keyword(s):  
Neural Networks ◽  
Light Field ◽  
Three Dimensional ◽  
Super Resolution ◽  
View Synthesis ◽  
Compact Representation ◽  
Light Weight ◽  
Computationally Efficient ◽  
Full Resolution ◽  
Time Systems

View synthesis (VS) for light field images is a very time-consuming task due to the great quantity of involved pixels and intensive computations, which may prevent it from the practical three-dimensional real-time systems. In this article, we propose an acceleration approach for deep learning-based light field view synthesis, which can significantly reduce calculations by using compact-resolution (CR) representation and super-resolution (SR) techniques, as well as light-weight neural networks. The proposed architecture has three cascaded neural networks, including a CR network to generate the compact representation for original input views, a VS network to synthesize new views from down-scaled compact views, and a SR network to reconstruct high-quality views with full resolution. All these networks are jointly trained with the integrated losses of CR, VS, and SR networks. Moreover, due to the redundancy of deep neural networks, we use the efficient light-weight strategy to prune filters for simplification and inference acceleration. Experimental results demonstrate that the proposed method can greatly reduce the processing time and become much more computationally efficient with competitive image quality.

scholarly journals Deep robust residual network for super-resolution of 2D fetal brain MRI

2022 ◽  
Vol 12 (1) ◽  
Author(s):  
Liyao Song ◽  
Quan Wang ◽  
Ting Liu ◽  
Haiwei Li ◽  
Jiancun Fan ◽  
...  
Keyword(s):  
Spatial Resolution ◽  
High Efficiency ◽  
Brain Mri ◽  
Fetal Brain ◽  
Super Resolution ◽  
Adult Brain ◽  
Superior Performance ◽  
Residual Learning ◽  
2D Data

AbstractSpatial resolution is a key factor of quantitatively evaluating the quality of magnetic resonance imagery (MRI). Super-resolution (SR) approaches can improve its spatial resolution by reconstructing high-resolution (HR) images from low-resolution (LR) ones to meet clinical and scientific requirements. To increase the quality of brain MRI, we study a robust residual-learning SR network (RRLSRN) to generate a sharp HR brain image from an LR input. Due to the Charbonnier loss can handle outliers well, and Gradient Difference Loss (GDL) can sharpen an image, we combined the Charbonnier loss and GDL to improve the robustness of the model and enhance the texture information of SR results. Two MRI datasets of adult brain, Kirby 21 and NAMIC, were used to train and verify the effectiveness of our model. To further verify the generalizability and robustness of the proposed model, we collected eight clinical fetal brain MRI 2D data for evaluation. The experimental results have shown that the proposed deep residual-learning network achieved superior performance and high efficiency over other compared methods.

Sign in / Sign up

Export Citation Format

Share Document