Semantic Segmentation with Light Field Imaging and Convolutional Neural Networks

2021 ◽  
pp. 1-1
Author(s):  
Chen Jia ◽  
Fan Shi ◽  
Meng Zhao ◽  
Yao Zhang ◽  
Xu Cheng ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Light Field ◽  
Semantic Segmentation ◽  
Light Field Imaging ◽  
Field Imaging

scholarly journals Depth Estimation from Light Field Geometry Using Convolutional Neural Networks

Sensors ◽  
2021 ◽  
Vol 21 (18) ◽  
pp. 6061
Author(s):  
Lei Han ◽  
Xiaohua Huang ◽  
Zhan Shi ◽  
Shengnan Zheng
Keyword(s):  
Neural Networks ◽  
Feature Extraction ◽  
Convolutional Neural Networks ◽  
Field Data ◽  
Light Field ◽  
Depth Estimation ◽  
Computational Time ◽  
Synthetic Image ◽  
Light Field Imaging ◽  
Field Imaging

Depth estimation based on light field imaging is a new methodology that has succeeded the traditional binocular stereo matching and depth from monocular images. Significant progress has been made in light-field depth estimation. Nevertheless, the balance between computational time and the accuracy of depth estimation is still worth exploring. The geometry in light field imaging is the basis of depth estimation, and the abundant light-field data provides convenience for applying deep learning algorithms. The Epipolar Plane Image (EPI) generated from the light-field data has a line texture containing geometric information. The slope of the line is proportional to the depth of the corresponding object. Considering the light field depth estimation as a spatial density prediction task, we design a convolutional neural network (ESTNet) to estimate the accurate depth quickly. Inspired by the strong image feature extraction ability of convolutional neural networks, especially for texture images, we propose to generate EPI synthetic images from light field data as the input of ESTNet to improve the effect of feature extraction and depth estimation. The architecture of ESTNet is characterized by three input streams, encoding-decoding structure, and skipconnections. The three input streams receive horizontal EPI synthetic image (EPIh), vertical EPI synthetic image (EPIv), and central view image (CV), respectively. EPIh and EPIv contain rich texture and depth cues, while CV provides pixel position association information. ESTNet consists of two stages: encoding and decoding. The encoding stage includes several convolution modules, and correspondingly, the decoding stage embodies some transposed convolution modules. In addition to the forward propagation of the network ESTNet, some skip-connections are added between the convolution module and the corresponding transposed convolution module to fuse the shallow local and deep semantic features. ESTNet is trained on one part of a synthetic light-field dataset and then tested on another part of the synthetic light-field dataset and real light-field dataset. Ablation experiments show that our ESTNet structure is reasonable. Experiments on the synthetic light-field dataset and real light-field dataset show that our ESTNet can balance the accuracy of depth estimation and computational time.

EPIModules on a Geodesic: Toward 360° Light-Field Imaging

2019 ◽  
Vol 2019 (3) ◽  
pp. 636-1-636-6
Author(s):  
H. Harlyn Baker ◽  
Gregorij Kurillo ◽  
Allan Miller ◽  
Alessandro Temil ◽  
Tom Defanti ◽  
...  
Keyword(s):  
Light Field ◽  
Light Field Imaging ◽  
Field Imaging

scholarly journals Semantic Segmentation of Cerebellum in 2D Fetal Ultrasound Brain Images using Convolutional Neural Networks

IEEE Access ◽  
2021 ◽  
pp. 1-1
Author(s):  
Vishal Singh ◽  
Pradeeba Sridar ◽  
Jinman Kim ◽  
Ralph Nanan ◽  
N. Poornima ◽  
...  
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Semantic Segmentation ◽  
Fetal Ultrasound ◽  
Brain Images

scholarly journals Convolutional Neural Networks for Semantic Segmentation as a Tool for Multiclass Face Analysis in Thermal Infrared

2021 ◽  
Vol 40 (1) ◽  
Author(s):  
David Müller ◽  
Andreas Ehlen ◽  
Bernd Valeske
Keyword(s):  
Neural Networks ◽  
Convolutional Neural Networks ◽  
Semantic Segmentation ◽  
Thermal Infrared ◽  
Relative Temperature ◽  
Face Analysis ◽  
Image Translation ◽  
The Face ◽  
Live Image ◽  
Quantitative Thermography

AbstractConvolutional neural networks were used for multiclass segmentation in thermal infrared face analysis. The principle is based on existing image-to-image translation approaches, where each pixel in an image is assigned to a class label. We show that established networks architectures can be trained for the task of multiclass face analysis in thermal infrared. Created class annotations consisted of pixel-accurate locations of different face classes. Subsequently, the trained network can segment an acquired unknown infrared face image into the defined classes. Furthermore, face classification in live image acquisition is shown, in order to be able to display the relative temperature in real-time from the learned areas. This allows a pixel-accurate temperature face analysis e.g. for infection detection like Covid-19. At the same time our approach offers the advantage of concentrating on the relevant areas of the face. Areas of the face irrelevant for the relative temperature calculation or accessories such as glasses, masks and jewelry are not considered. A custom database was created to train the network. The results were quantitatively evaluated with the intersection over union (IoU) metric. The methodology shown can be transferred to similar problems for more quantitative thermography tasks like in materials characterization or quality control in production.

Sign in / Sign up

Export Citation Format

Share Document