Depth Measurement and 3D Reconstruction of Stereo Images

Abstract. The paper presents our efforts on CNN-based 3D reconstruction of the Martian surface using monocular images. The Viking colorized global mosaic and Mar Express HRSC blended DEM are used as training data. An encoder-decoder network system is employed in the framework. The encoder section extracts features from the images, which includes convolution layers and reduction layers. The decoder section consists of deconvolution layers and is to integrate features and convert the images to desired DEMs. In addition, skip connection between encoder and decoder section is applied, which offers more low-level features for the decoder section to improve its performance. Monocular Context Camera (CTX) images are used to test and verify the performance of the proposed CNN-based approach. Experimental results show promising performances of the proposed approach. Features in images are well utilized, and topographical details in images are successfully recovered in the DEMs. In most cases, the geometric accuracies of the generated DEMs are comparable to those generated by the traditional technology of photogrammetry using stereo images. The preliminary results show that the proposed CNN-based approach has great potential for 3D reconstruction of the Martian surface.

Download Full-text

From 2D Images to 3D Tangible Models: Autostereoscopic and Haptic Visualization of Martian Rocks in Virtual Environments

Presence Teleoperators & Virtual Environments ◽

10.1162/pres.16.1.1 ◽

2007 ◽

Vol 16 (1) ◽

pp. 1-15 ◽

Cited By ~ 3

Author(s):

Cagatay Basdogan

Keyword(s):

3D Reconstruction ◽

3D Models ◽

Rock Surface ◽

Surface Model ◽

Stereo Images ◽

Martian Surface ◽

Reconstruction Process ◽

Low Transmission ◽

2D Images ◽

Multimodal Visualization

A planetary rover acquires a large collection of images while exploring its surrounding environment. For example, 2D stereo images of the Martian surface captured by the lander and the Sojourner rover during the Mars Pathfinder mission in 1997 were transmitted to Earth for scientific analysis and navigation planning. Due to the limited memory and computational power of the Sojourner rover, most of the images were captured by the lander and then transmitted to Earth directly for processing. If these images were merged together at the rover site to reconstruct a 3D representation of the rover's environment using its on-board resources, more information could potentially be transmitted to Earth in a compact manner. However, construction of a 3D model from multiple views is a highly challenging task to accomplish even for the new generation rovers (Spirit and Opportunity) running on the Mars surface at the time this article was written. Moreover, low transmission rates and communication intervals between Earth and Mars make the transmission of any data more difficult. We propose a robust and computationally efficient method for progressive transmission of multi-resolution 3D models of Martian rocks and soil reconstructed from a series of stereo images. For visualization of these models on Earth, we have developed a new multimodal visualization setup that integrates vision and touch. Our scheme for 3D reconstruction of Martian rocks from 2D images for visualization on Earth involves four main steps: a) acquisition of scans: depth maps are generated from stereo images, b) integration of scans: the scans are correctly positioned and oriented with respect to each other and fused to construct a 3D volumetric representation of the rocks using an octree, c) transmission: the volumetric data is encoded and progressively transmitted to Earth, d) visualization: a surface model is reconstructed from the transmitted data on Earth and displayed to a user through a new autostereoscopic visualization table and a haptic device for providing touch feedback. To test the practical utility of our approach, we first captured a sequence of stereo images of a rock surface from various viewpoints in JPL MarsYard using a mobile cart and then performed a series of 3D reconstruction experiments. In this paper, we discuss the steps of our reconstruction process, our multimodal visualization system, and the tradeoffs that have to be made to transmit multiresolution 3D models to Earth in an efficient manner under the constraints of limited computational resources, low transmission rate, and communication interval between Earth and Mars.

Download Full-text

3D Reconstruction of Point Clouds and Solid Models from LiDAR and Digital Stereo Images

4th International Symposium on Innovative Approaches in Engineering and Natural Sciences Proceedings ◽

10.36287/setsci.4.6.059 ◽

2019 ◽

Author(s):

Abu Kamara ◽

Sedat Doğan

Keyword(s):

3D Reconstruction ◽

Point Clouds ◽

Stereo Images ◽

Solid Models

Download Full-text

Real-Time 3D Reconstruction Method Based on Monocular Vision

Sensors ◽

10.3390/s21175909 ◽

2021 ◽

Vol 21 (17) ◽

pp. 5909

Author(s):

Qingyu Jia ◽

Liang Chang ◽

Baohua Qiang ◽

Shihao Zhang ◽

Wu Xie ◽

...

Keyword(s):

3D Reconstruction ◽

Real Time ◽

Visual Information ◽

Three Dimensional ◽

Position Estimation ◽

Monocular Vision ◽

Cluster Center ◽

Reconstruction Method ◽

The Real ◽

Depth Measurement

Real-time 3D reconstruction is one of the current popular research directions of computer vision, and it has become the core technology in the fields of virtual reality, industrialized automatic systems, and mobile robot path planning. Currently, there are three main problems in the real-time 3D reconstruction field. Firstly, it is expensive. It requires more varied sensors, so it is less convenient. Secondly, the reconstruction speed is slow, and the 3D model cannot be established accurately in real time. Thirdly, the reconstruction error is large, which cannot meet the requirements of scenes with accuracy. For this reason, we propose a real-time 3D reconstruction method based on monocular vision in this paper. Firstly, a single RGB-D camera is used to collect visual information in real time, and the YOLACT++ network is used to identify and segment the visual information to extract part of the important visual information. Secondly, we combine the three stages of depth recovery, depth optimization, and deep fusion to propose a three-dimensional position estimation method based on deep learning for joint coding of visual information. It can reduce the depth error caused by the depth measurement process, and the accurate 3D point values of the segmented image can be obtained directly. Finally, we propose a method based on the limited outlier adjustment of the cluster center distance to optimize the three-dimensional point values obtained above. It improves the real-time reconstruction accuracy and obtains the three-dimensional model of the object in real time. Experimental results show that this method only needs a single RGB-D camera, which is not only low cost and convenient to use, but also significantly improves the speed and accuracy of 3D reconstruction.

Download Full-text

Robust Detection of Point Correspondences in Stereo Images

Acta Polytechnica ◽

10.14311/956 ◽

2007 ◽

Vol 47 (4-5) ◽

Author(s):

A. Stojanovic ◽

M. Unger

Keyword(s):

3D Reconstruction ◽

Fundamental Matrix ◽

Probabilistic Approach ◽

Geometric Constraint ◽

Stereo Images ◽

Automatic Computation ◽

Robust Detection ◽

Image Pairs ◽

New Criterion

A major challenge in 3D reconstruction is the computation of the fundamental matrix. Automatic computation from uncalibrated image pairs is performed from point correspondences. Due to imprecision and wrong correspondences, only an approximation of the true fundamental matrix can be computed. The quality of the fundamental matrix strongly depends on the location and number of point correspondences.Furthermore, the fundamental matrix is the only geometric constraint between two uncalibrated views, and hence it can be used for the detection of wrong point correspondences. This property is used by current algorithms like RANSAC, which computes the fundamental matrix from a restricted set of point correspondences. In most cases, not only wrong correspondences are disregarded, but also correct ones, which is due to the criterion used to eliminate outliers. In this context, a new criterion preserving a maximum of correct correspondences would be useful.In this paper we introduce a novel criterion for outlier elimination based on a probabilistic approach. The enhanced set of correspondences may be important for further computation towards a 3D reconstruction of the scene.

Download Full-text