scholarly journals High-Fidelity Depth Upsampling Using the Self-Learning Framework

Sensors ◽  
2018 ◽  
Vol 19 (1) ◽  
pp. 81
Author(s):  
Inwook Shim ◽  
Tae-Hyun Oh ◽  
In Kweon
Keyword(s):  
Stereo Matching ◽  
Depth Map ◽  
The Self ◽  
Sensor Data ◽  
High Fidelity ◽  
Learning Framework ◽  
Depth Maps ◽  
Potential Applications ◽  
Self Learning

This paper presents a depth upsampling method that produces a high-fidelity dense depth map using a high-resolution RGB image and LiDAR sensor data. Our proposed method explicitly handles depth outliers and computes a depth upsampling with confidence information. Our key idea is the self-learning framework, which automatically learns to estimate the reliability of the upsampled depth map without human-labeled annotation. Thereby, our proposed method can produce a clear and high-fidelity dense depth map that preserves the shape of object structures well, which can be favored by subsequent algorithms for follow-up tasks. We qualitatively and quantitatively evaluate our proposed method by comparing other competing methods on the well-known Middlebury 2014 and KITTIbenchmark datasets. We demonstrate that our method generates accurate depth maps with smaller errors favorable against other methods while preserving a larger number of valid points, as we also show that our approach can be seamlessly applied to improve the quality of depth maps from other depth generation algorithms such as stereo matching and further discuss potential applications and limitations. Compared to previous work, our proposed method has similar depth errors on average, while retaining at least 3% more valid depth points.

scholarly journals The Design and Implementation of Postprocessing for Depth Map on Real-Time Extraction System

2014 ◽  
Vol 2014 ◽  
pp. 1-10 ◽  
Author(s):  
Zhiwei Tang ◽  
Bin Li ◽  
Huosheng Li ◽  
Zheng Xu
Keyword(s):  
Real Time ◽  
Stereo Matching ◽  
Depth Map ◽  
Depth Estimation ◽  
Consistency Check ◽  
Key Technology ◽  
Design And Implementation ◽  
Depth Maps ◽  
Matching Cost

Depth estimation becomes the key technology to resolve the communications of the stereo vision. We can get the real-time depth map based on hardware, which cannot implement complicated algorithm as software, because there are some restrictions in the hardware structure. Eventually, some wrong stereo matching will inevitably exist in the process of depth estimation by hardware, such as FPGA. In order to solve the problem a postprocessing function is designed in this paper. After matching cost unique test, the both left-right and right-left consistency check solutions are implemented, respectively; then, the cavities in depth maps can be filled by right depth values on the basis of right-left consistency check solution. The results in the experiments have shown that the depth map extraction and postprocessing function can be implemented in real time in the same system; what is more, the quality of the depth maps is satisfactory.

scholarly journals No-Reference Depth Map Quality Evaluation Model Based on Depth Map Edge Confidence Measurement in Immersive Video Applications

2019 ◽  
Vol 11 (10) ◽  
pp. 204 ◽  
Author(s):  
Dogan ◽  
Haddad ◽  
Ekmekcioglu ◽  
Kondoz
Keyword(s):  
Quality Evaluation ◽  
Evaluation Model ◽  
Depth Map ◽  
Depth Image ◽  
Depth Maps ◽  
Model Based ◽  
Objective Quality ◽  
Quality Evaluation Model ◽  
Confidence Measurement

When it comes to evaluating perceptual quality of digital media for overall quality of experience assessment in immersive video applications, typically two main approaches stand out: Subjective and objective quality evaluation. On one hand, subjective quality evaluation offers the best representation of perceived video quality assessed by the real viewers. On the other hand, it consumes a significant amount of time and effort, due to the involvement of real users with lengthy and laborious assessment procedures. Thus, it is essential that an objective quality evaluation model is developed. The speed-up advantage offered by an objective quality evaluation model, which can predict the quality of rendered virtual views based on the depth maps used in the rendering process, allows for faster quality assessments for immersive video applications. This is particularly important given the lack of a suitable reference or ground truth for comparing the available depth maps, especially when live content services are offered in those applications. This paper presents a no-reference depth map quality evaluation model based on a proposed depth map edge confidence measurement technique to assist with accurately estimating the quality of rendered (virtual) views in immersive multi-view video content. The model is applied for depth image-based rendering in multi-view video format, providing comparable evaluation results to those existing in the literature, and often exceeding their performance.

Hardware Feedback Self-Organizing Map and its Application to Mobile Robot Location Identification

2007 ◽  
Vol 11 (8) ◽  
pp. 937-945 ◽  
Author(s):  
Hiroomi Hikawa ◽  
◽  
Kazutoshi Harada ◽  
Takenori Hirabayashi ◽  
Keyword(s):  
Mobile Robot ◽  
Weight Vector ◽  
Hardware Architecture ◽  
The Self ◽  
Sensor Data ◽  
Parallel Structure ◽  
Self Organizing Map ◽  
Cluster Data ◽  
Self Learning ◽  
Self Organizing

We propose new hardware architecture for the self-organizing map (SOM) and feedback SOM (FSOM). Due to the parallel structure in the SOM and FSOM algorithm, customized hardware considerably speeds-up processing. Proposed hardware FSOM identifies the location of a mobile robot from a sequence of direction data. The FSOM is self-trained to cluster data to identify where the robot is. The proposed FSOM design is described in C and VHDL, and its performance is tested by simulation using actual sensor data from an experimental mobile robot. Results show that the hardware FSOM succeeds in self-learning to find the robot’s location. The hardware FSOM is estimated to process 6,992 million weight-vector elements per second.

scholarly journals Development of an instructor guide tool: ‘Three Stages of Holistic Debriefing’

2020 ◽  
Vol 28 ◽  
Author(s):  
Fernanda dos Santos Nogueira de Góes ◽  
Deirdre Jackman
Keyword(s):  
Literature Review ◽  
Reflective Learning ◽  
High Fidelity ◽  
Tool Development ◽  
Pedagogical Practice ◽  
High Fidelity Simulation ◽  
Educator Training ◽  
Learning Framework ◽  
Nursing Educator

Objective: to describe the development of an English and Brazilian Portuguese Holistic Debriefing Tool focused on nursing educator to promote a reflective learning. Method: a methodology study, with three phases: integrative literature review; tool development and review of a panel of nursing experts. The literature review tracked a systematic process. For the tool development were used literature review results, Lederman’s Debriefing Process and Zabala’s learning framework as theoretical referential to promote a reflective learning in High-Fidelity Simulation. The panel of nursing experts analysed the quality of the tool. Results: literature review evidenced gaps about educator pedagogical preparation and indicated no holistic debriefing tool exists which captures formative and summative aspects of debriefing guidance to assist the educator to debrief. Debriefing tool was purposed with two pages: first page were recommended how conduct debriefing and second page is a questions guidance. The tool evaluation was undertaken for a total of three modifications for congruence and concept reader clarity. Conclusion: it was proposed a holistic debriefing tool focused on nursing educator. This study provides an overall picture of the process to promote a reflexive learning in High-Fidelity Simulation and to contribute to formal nursing educator training to apply best pedagogical practice.

scholarly journals Algorithm for correcting the image depth map based on the points brightness and their distance from the observation point

2020 ◽  
Vol 47 (3) ◽  
pp. 82-92
Author(s):  
S. I. Korotkevich ◽  
Yu. V. Minaeva
Keyword(s):  
Stereo Vision ◽  
Depth Map ◽  
Human Head ◽  
Observation Point ◽  
Mathematical Software ◽  
Depth Maps ◽  
Trained Personnel ◽  
Mathematical Algorithms ◽  
Image Contour

Objective. Modeling the human head is a significant problem that arises in a wide variety of fields of science and technology. Existing active technologies for reconstruction and modeling of the object under study require expensive equipment and trained personnel. Methods. An alternative is to use passive methods that perform image processing using special mathematical algorithms. One of these methods is the stereo vision, which is based on the use of paired images taken simultaneously with several cameras positioned and calibrated in a certain way. However, a common drawback of stereo vision methods is the possibility of obtaining erroneous depth maps due to poorquality source images or incorrect camera and lighting settings. Results. Procedures were developed that use additional parameters of image points, which can be used to correct depth maps to avoid the appearance of defects. To achieve this objective, the existing mathematical software for processing photo and video materials is analyzed; methods for suppressing noise in the image, obtaining an image contour, as well as a method for obtaining a 3D object matrix based on changing the direction of illumination are proposed; the algorithm is tested on a test example. Conclusion. The developed technique should improve the quality of the depth map of the processed image and thus make the modeling procedures more efficient. 

scholarly journals ENHANCEMENT OF DEPTH MAP BY FUSION USING ADAPTIVE AND SEMANTIC-GUIDED SPATIOTEMPORAL FILTERING

2020 ◽  
Vol V-3-2020 ◽  
pp. 227-232
Author(s):  
H. Albanwan ◽  
R. Qin
Keyword(s):  
Depth Map ◽  
Spatiotemporal Analysis ◽  
Stereo Image ◽  
Single Pair ◽  
Depth Maps ◽  
3D Model Reconstruction ◽  
Digital Elevation ◽  
Image Pairs ◽  
Spatiotemporal Filters

Abstract. Extracting detailed geometric information about a scene relies on the quality of the depth maps (e.g. Digital Elevation Surfaces, DSM) to enhance the performance of 3D model reconstruction. Elevation information from LiDAR is often expensive and hard to obtain. The most common approach to generate depth maps is through multi-view stereo (MVS) methods (e.g. dense stereo image matching). The quality of single depth maps, however, is often prone to noise, outliers, and missing data points due to the quality of the acquired image pairs. A reference multi-view image pair must be noise-free and clear to ensure high-quality depth maps. To avoid such a problem, current researches are headed toward fusing multiple depth maps to recover the shortcomings of single-depth maps resulted from a single pair of multi-view images. Several approaches tackled this problem by merging and fusing depth maps, using probabilistic and deterministic methods, but few discussed how these fused depth maps can be refined through adaptive spatiotemporal analysis algorithms (e.g. spatiotemporal filters). The motivation is to push towards preserving the high precision and detail level of depth maps while optimizing the performance, robustness, and efficiency of the algorithm.

scholarly journals Depth Map Renement Using Reliability Based Joint Trilateral Filter

1970 ◽  
Vol 7 (2) ◽  
pp. 107-116
Author(s):  
Takuya Matsuo ◽  
Naoki Kodera ◽  
Norishige Fukushima ◽  
Yutaka Ishibashi
Keyword(s):  
Lower Cost ◽  
Computational Cost ◽  
Depth Map ◽  
Optimization Methods ◽  
Depth Image ◽  
Depth Maps ◽  
Trilateral Filter ◽  
Depth Image Based Rendering ◽  
Depth Ranges

In this paper, we propose a renement lter for depth maps. The lter convolutes an image and a depth map with a cross computed kernel. We call the lter joint trilateral lter. Main advantages of the proposed method are that the lter ts outlines of objects in the depth map to silhouettes in the im- age, and the lter reduces Gaussian noise in other areas. The eects reduce rendering artifacts when a free viewpoint image is generated by point cloud ren- dering and depth image based rendering techniques. Additionally, their computational cost is independent of depth ranges. Thus we can obtain accurate depth maps with the lower cost than the conventional ap- proaches, which require Markov random eld based optimization methods. Experimental results show that the accuracy of the depth map in edge areas goes up and its running time decreases. In addition, the lter improves the accuracy of edges in the depth map from Kinect sensor. As results, the quality of the rendering image is improved.

scholarly journals Depth Map Quality Evaluation for Photographic Applications

2020 ◽  
Vol 2020 (9) ◽  
pp. 370-1-370-7
Author(s):  
Eloi Zalczer ◽  
François-Xavier Thomas ◽  
Laurent Chanas ◽  
Gabriele Facciolo ◽  
Frédéric Guichard
Keyword(s):  
Augmented Reality ◽  
Quality Evaluation ◽  
Depth Map ◽  
Depth Imaging ◽  
Depth Maps ◽  
Spatial Alignment ◽  
Need To Evaluate ◽  
New Challenges ◽  
Consumer Devices

As depth imaging is integrated into more and more consumer devices, manufacturers have to tackle new challenges. Applica- tions such as computational bokeh and augmented reality require dense and precisely segmented depth maps to achieve good re- sults. Modern devices use a multitude of different technologies to estimate depth maps, such as time-of-flight sensors, stereoscopic cameras, structured light sensors, phase-detect pixels or a com- bination thereof. Therefore, there is a need to evaluate the quality of the depth maps, regardless of the technology used to produce them. The aim of our work is to propose an end-result evalua- tion method based on a single scene, using a specifically designed chart. We consider the depth maps embedded in the photographs, which are not visible to the user but are used by specialized soft- ware, in association with the RGB pictures. Some of the aspects considered are spatial alignment between RGB and depth, depth consistency, and robustness to texture variations. This work also provides a comparison of perceptual and automatic evaluations.

scholarly journals PENGEMBANGAN MODUL FISIKA MATERI GERAK PARABOLA BERBASIS GENERATIVE LEARNING

2019 ◽  
Vol 4 ◽  
pp. 186
Author(s):  
Destia Mareta Dyah Santoso ◽  
Winarti Winarti
Keyword(s):  
Learning Strategy ◽  
Excellent Result ◽  
The Self ◽  
Average Score ◽  
Generative Learning ◽  
Test Technique ◽  
Physics Teachers ◽  
Study Results ◽  
Self Learning

<p class="AbstractEnglish"><strong>Abstract:</strong>. Generative learning is a learning strategy with constructivism approach, where the students have the opportunity to construct their own knowledge. The aims of this study is 1) to design a physics module based on generative learning for the topic of parabolic motion. 2) to know the quality of the physics module based on generative learning for the topic of parabolic motion. 3) to know students responses about the developed physics module. This study is an R&amp;D research with procedures which adapts the development procedures of the 4D model, this model consists of define, design, develop, and disseminate. The data collecting technique in this study is the non-test technique with a questionnaires method. The instruments which used are validation sheets, module evaluation sheets, and students responses questionnaires. The result of this study are 1) it has been developed a physics module for the topic of parabolic motion based on generative learning 2) the quality oh physics module based on generative learning, based on the assessment of physics material experts, media experts, physics teachers has the excellent result with an average score 3,44; 3,66; and 3,64. And 3) the student’s responses in a limited test show that the average students give agreement to the developed product with average score 0,95. These study results show that a physics module based on generative learning for parabolic motion topic is suitable yo use for one of the self-learning references.</p><p class="AbstrakIndonesia"><strong>Abstrak:</strong> <em>Generative Learning</em> merupakan strategi pembelajaran dengan pendekatan kontruktivisme, dimana peserta didik dapat memperoleh kesempatan untuk mengkontruksi pengetahuannya sendiri. Tujuan penelitian ini adalah untuk 1) mendesain modul fisika berbasis <em>generative learning</em> pada materi pokok gerak parabola; 2) mengetahui kualitas modul fisika berbasis <em>generative learning</em> pada materi pokok gerak parabola; 3) mengetahui respon peserta didik terhadap modul fisika yang telah dikembangkan. Penelitian ini merupakan penelitian R &amp; D dengan prosedural yang mengadaptasi prosedur pengembangan perangkat model 4-D, yakni <em>define, design, develop,</em><em> </em>dan<em> disseminate</em>. Teknik pengumpulan data penelitian ini adalah teknik non tes dengan metode angket. Adapun instrumen yang digunakan berupa lembar validasi, lembar penilaian modul, dan angket respon peserta didik. Hasil penelitian ini antara lain: 1) telah dihasilkan modul fisika materi gerak parabola berbasis <em>generative learning</em>; 2) kualitas modul fisika berbasis <em>generative learning</em> berdasarkan penilaian ahli materi, ahli media, dan guru fisika memiliki kategori Sangat Baik dengan skor rerata berturut-turut 3,44; 3,66; dan 3,64; dan 3) respon peserta didik pada uji terbatas menunjukkan bahwa rata-rata peserta didik menyatakan Setuju dengan adanya produk yang dikembangkan dengan skor rerata 0,98 dan respon peserta didik pada uji luas menunjukkan bahwa peserta didik menyatakan Setuju dengan adanya produk yang dikembangkan dengan skor rerata 0,95. Hasil penelitian ini menunjukkan bahwa modul fisika berbasis <em>generative learning</em> pada materi gerak parabola layak dijadikan sebagai salah satu sumber belajar mandiri.</p>

scholarly journals Learning About the Self: Motives for coherence and positivity constrain learning from self-relevant feedback

2021 ◽  
Author(s):  
Jacob Elder ◽  
Tyler Davis ◽  
Brent Hughes
Keyword(s):  
Reinforcement Learning ◽  
Negative Feedback ◽  
Positive Feedback ◽  
Mechanistic Explanation ◽  
The Self ◽  
Self Concept ◽  
Causal Network ◽  
Social Feedback ◽  
Learning Framework ◽  
Self Learning

People learn about themselves from social feedback, but desires for coherence and positivity constrain how feedback is incorporated into the self-concept. We develop a network-based model of the self-concept and embed it in a reinforcement learning framework to provide a mechanistic account of how motivations shape self-learning from feedback. Participants (n = 46) received feedback while self-evaluating on traits drawn from a causal network of trait semantics. Network-defined communities were assigned different likelihoods of positive feedback. Participants learned from positive feedback but dismissed negative feedback, as reflected by asymmetries in computational parameters that represent the incorporation of positive versus negative outcomes. Furthermore, participants were constrained in how they incorporated feedback: self-evaluations changed less for traits more important to coherence of the network. We provide a mechanistic explanation of how motives for coherence and positivity jointly constrain learning about the self from feedback that makes testable predictions for future clinical research.

Sign in / Sign up

Export Citation Format

Share Document