ReS2tAC—UAV-Borne Real-Time SGM Stereo Optimized for Embedded ARM and CUDA Devices

With the emergence of low-cost robotic systems, such as *UAV, the importance of embedded high-performance image processing has increased. For a long time, FPGAs were the only processing hardware that were capable of high-performance computing, while at the same time preserving a low power consumption, essential for embedded systems. However, the recently increasing availability of embedded GPU-based systems, such as the NVIDIA Jetson series, comprised of an ARM CPU and a NVIDIA Tegra GPU, allows for massively parallel embedded computing on graphics hardware. With this in mind, we propose an approach for real-time embedded stereo processing on ARM and CUDA-enabled devices, which is based on the popular and widely used Semi-Global Matching algorithm. In this, we propose an optimization of the algorithm for embedded CUDA GPUs, by using massively parallel computing, as well as using the NEON intrinsics to optimize the algorithm for vectorized SIMD processing on embedded ARM CPUs. We have evaluated our approach with different configurations on two public stereo benchmark datasets to demonstrate that they can reach an error rate as low as 3.3%. Furthermore, our experiments show that the fastest configuration of our approach reaches up to 46 FPS on VGA image resolution. Finally, in a use-case specific qualitative evaluation, we have evaluated the power consumption of our approach and deployed it on the DJI Manifold 2-G attached to a DJI Matrix 210v2 RTK *UAV, demonstrating its suitability for real-time stereo processing onboard a *UAV.

The Hayabusa2 lander MASCOT on the surface of asteroid (162173) Ryugu – Stereo-photogrammetric analysis of MASCam image data

After its release and a descent and bouncing phase, the Hayabusa2 lander MASCOT came to a final rest and MASCOT’s camera MASCam acquired a set of images of the surface of Ryugu. With MASCam’s instantaneous field of view of about 1 mrad, the images provide pixel scales from 0.2 to 0.5 mm pixel−1 in the foreground and up to 1 cm pixel−1 for surface parts in the background. Using a stereo-photogrammetric analysis of the MASCam images taken from slightly different positions due to commanded and unintentional movements of the MASCOT lander, we were able to determine the orientation for the different measurement positions. Furthermore, we derived a 3D surface model of MASCOT’s vicinity. Although the conditions for 3D stereo processing were poor due to very small stereo angles, the derived 3D model has about 0.5 cm accuracy in the foreground at 20 cm distance and about 1.5 cm at a distance of 40–50 cm.

EVALUATION OF SELECTED COST AGGREGATION METHODS ON HIGH RESOLUTION SATELLITE STEREO IMAGES

Dense stereo processing requires a critical step that called cost aggregation or cost optimization. Most of the cost aggregation methods are evaluated on close range stereo images from Middlebury or KITTI datasets. While the effect of cost aggregation on high resolution satellite stereo processing has not yet been sufficiently evaluated. In this paper, three typical cost aggregation methods together with another approach which is a combination of these methods are evaluated on high resolution satellite stereo images and then are compared with LiDAR ground truth. These methods including Semi-Global Matching (SGM), Guided Filtering (GF), iterative GF (IGF), and SGM followed by GF (SGM-GF) with Census and Zero Normalized Cross Correlation (ZNCC) cost functions. Although the Census cost function has a good performance on the border of the objects and low blurring effects, the results of both cost functions, i.e. Census and ZNCC, have same treatment on all stereo methods. Also, in order to make an impartial assessment, for all stereo methods, we do not perform any disparity map refinement. The bad-pixel criteria with an absolute difference height error greater than 2 meters for SGM, GF, IGF, and SGM-GF methods is 36.7%, 34.8%, 33.8%, and 28.6% respectively. Also, the Normalized Median Absolute Difference (NMAD) error for SGM, GF, IGF, and SGM-GF is 1.29, 1.15, 1.06, and 0.94 meters, respectively. Overall, the experimental results on WV III stereo images demonstrate that the SGM method has lower accuracy and SGM-GF method is accurate than other methods.

Implementation of a Self-Consistent Stereo Processing Chain for 3D Stereo Reconstruction of the Lunar Landing Sites

The department for Planetary Geodesy at Technical University Berlin is developing routines for photogrammetric processing of planetary image data to derive 3D representations of planetary surfaces. The Integrated Software for Imagers and Spectrometers (ISIS) software (Anderson et al., 2004), developed by USGS, Flagstaff, is readily available, open source, and very well documented. Hence, ISIS was chosen as a prime processing platform and tool kit. However, ISIS does not provide a full photogrammetric stereo processing chain. Several components like image matching, bundle block adjustment (until recently) or digital terrain model (DTM) interpolation from 3D object points are missing. Our group aims to complete this photogrammetric stereo processing chain by implementing the missing components, taking advantage of already existing ISIS classes and functionality. We report here on the current status of the development of our stereo processing chain and its first application on the Lunar Apollo landing sites.

Establishment of Cutting Model for Three-Axis Surface Machining Based on SolidCAM

For different three-axis machining characteristics, SolidCAM provides many machining strategies including linear, correction, spiral, imitation cutting and so on. This paper chooses three-axis revolving curved surface parts as the research object and reference three-axis stereo processing combinating wrap-around roughing and linear processing. Finally, the cutting model of this machining strategy based on parameters has been established to provide a reference for the determination of cutting parameters.