Enhancement of light field disparity maps by reducing the silhouette effect and plane noise

During the last decade, there has been an increasing number of applications dealing with multidimensional visual information, either for 3D object representation or feature extraction purposes. In this context, recent advances in light field technology, have been driving research efforts in disparity estimation methods. Among the existing ones, those based on the structure tensor have emerged as very promising to estimate disparity maps from Epipolar Plane Images. However, this approach is known to have two intrinsic limitations: (i) silhouette enlargement and (ii) irregularity of surface normal maps as computed from the estimated disparity. To address these problems, this work proposes a new method for improving disparity maps obtained from the structure-tensor approach by enhancing the silhouette and reducing the noise of planar surfaces in light fields. An edge-based approach is initially used for silhouette improvement through refinement of the estimated disparity values around object edges. Then, a plane detection algorithm, based on a seed growth strategy, is used to estimate planar regions, which in turn are used to guide correction of erroneous disparity values detected in object boundaries. The proposed algorithm shows an average improvement of 98.3% in terms of median angle error for plane surfaces, when compared to regular structure-tensor-based methods, outperforming state-of-the-art methods. The proposed framework also presents very competitive results, in terms of mean square error between disparity maps and their ground truth, when compared with their counterparts.

Joint Roughness Profiling using Photogrammetry

We propose an automated camera setup for photogrammetric roughness analysis in the laboratory environment. The developed fast and low-cost automation setup can be very useful for tedious and laborsome manual field logging practices. The photographs are processed in MATLAB to obtain disparity maps. Coding routines for stereo photogrammetry and digital measurements are written in MATLAB. Secondly, 6 effecting factors (projecting an image onto core face, depth of field, brightness, camera-to-object to baseline distance ratio, projected image size and occlusion) influencing noise in roughness depth maps computed by employing stereo photogrammetry are investigated. After deciding the best values that allow the lowest amount of noise, depth maps of 6 core faces are computed. Using the 3D point cloud generated, roughness profile measurements are made. Then, 8 profile measurements are made for each core face, both manually and digitally. The accuracy of the disparity maps has been verified by comparing 48 joint roughness coefficient (JRC) measurements made manually using a profile gauge. It was proved that surface roughness can be measured very fast in millimetric accuracy with an average Root Mean Square Error (RMSE) of 3.50 and Mean Absolute Error (MAE) of 3.02 by the help of the proposed set-up and calibration.

Learned Collaborative Stereo Refinement

In this work, we propose a learning-based method to denoise and refine disparity maps. The proposed variational network arises naturally from unrolling the iterates of a proximal gradient method applied to a variational energy defined in a joint disparity, color, and confidence image space. Our method allows to learn a robust collaborative regularizer leveraging the joint statistics of the color image, the confidence map and the disparity map. Due to the variational structure of our method, the individual steps can be easily visualized, thus enabling interpretability of the method. We can therefore provide interesting insights into how our method refines and denoises disparity maps. To this end, we can visualize and interpret the learned filters and activation functions and prove the increased reliability of the predicted pixel-wise confidence maps. Furthermore, the optimization based structure of our refinement module allows us to compute eigen disparity maps, which reveal structural properties of our refinement module. The efficiency of our method is demonstrated on the publicly available stereo benchmarks Middlebury 2014 and Kitti 2015.

Stereo Matching with Spatiotemporal Disparity Refinement Using Simple Linear Iterative Clustering Segmentation

Stereo matching is a challenging problem, especially for computer vision, e.g., three-dimensional television (3DTV) or 3D visualization. The disparity maps from the video streams must be estimated. However, the estimated disparity sequences may cause undesirable flickering errors. These errors result in poor visual quality for the synthesized video and reduce the video coding information. In order to solve this problem, we here propose a spatiotemporal disparity refinement method for local stereo matching using the simple linear iterative clustering (SLIC) segmentation strategy, outlier detection, and refinements of the temporal and spatial domains. In the outlier detection, the segmented region in the initial disparity is used to distinguish errors in the binocular disparity. Based on the color similarity and disparity difference, we recalculate the aggregated cost to determine adaptive disparities to recover the disparity errors in disparity sequences. The flickering errors are also effectively removed, and the object boundaries are well preserved. Experiments using public datasets demonstrated that our proposed method creates high-quality disparity maps and obtains a high peak signal-to-noise ratio compared to state-of-the-art methods.

Application of morphological processing methods in computer stereo vision

Computer vision algorithms are important for many areas of human activity. In particular, the number of applications related to the need to process images of real-world objects with computerized tools and the subsequent use of descriptive information in a variety of interactive and automated decision-making systems is increased. An important tool for analyzing real-world scenes are approaches to the application of stereo vision algorithms. The important step of many stereo matching algorithms is a disparity map. Depending on the content of the observed scene, part of the values on the disparity map can be immediately attributed to background values on a certain basis, or form a "natural" background, which is characterized by loss of informative data due to unacceptable error of subsequent resultant distance values. The calculated disparity map of any algorithm may contain some shortcomings in the form of discontinuities of continuous information areas caused by the complexity of shooting conditions, the impact of noise of various natures, hardware imperfections, and so on. An approach to mitigating the undesirable influence of negative factors on the resulting disparity is the use of mathematical morphology operations to process disparity maps at the post-processing stage. This paper presents information technology for increasing the content of disparity maps based on the mathematical morphology methods. The technology is based on a combination of morphological operations of erosion and dilation, which eliminates the typical problems of discontinuities of monotone regions and erroneous values on disparity maps. The proposed approach allows reducing the impact of common problems that arise during the operation of stereo matching algorithms, as well as increase the overall informativeness of disparity maps for images of real objects in the absence of partial or complete initial data on the characteristics of the observed scene. The results of testing morphological operations with disparity maps for real objects allow us to conclude about the possibility of partial restoration of areas of disparity maps with gaps in continuous information areas, as well as to reduce the impact of random anomalous values on the overall content of the disparity maps.

Image Distortion and Rectification Calibration Algorithms and Validation Technique for a Stereo Camera

This paper focuses on the calibration problem using stereo camera images. Currently, advanced vehicle systems such as smart cars and mobile robots require accurate and reliable vision in order to detect obstacles and special marks around. Such modern vehicles can be equipped with sensors and cameras together or separately. In this study, we propose new methodologies of stereo camera calibration based on the correction of distortion and image rectification. Once the calibration is complete, the validation of the corrections is presented followed by an evaluation of the calibration process. Usually, the validation section is not jointly considered with the calibration in other studies. However, the mass production of cameras widely uses the validation techniques in calibrations owned by manufacturing businesses. Here, we aim to present a single process for the calibration and validation of stereo cameras. The experiment results showed the disparity maps in comparison with another study and proved that the proposed calibration methods can be efficient.