A Deep Neural Network Sensor for Visual Servoing in 3D Spaces

This paper describes a novel stereo vision sensor based on deep neural networks, that can be used to produce a feedback signal for visual servoing in unmanned aerial vehicles such as drones. Two deep convolutional neural networks attached to the stereo camera in the drone are trained to detect wind turbines in images and stereo triangulation is used to calculate the distance from a wind turbine to the drone. Our experimental results show that the sensor produces data accurate enough to be used for servoing, even in the presence of noise generated when the drone is not being completely stable. Our results also show that appropriate filtering of the signals is needed and that to produce correct results, it is very important to keep the wind turbine within the field of vision of both cameras, so that both deep neural networks could detect it.

Measurement of Dynamic Responses from Large Structural Tests by Analyzing Non-Synchronized Videos

Image analysis techniques have been employed to measure displacements, deformation, crack propagation, and structural health monitoring. With the rapid development and wide application of digital imaging technology, consumer digital cameras are commonly used for making such measurements because of their satisfactory imaging resolution, video recording capability, and relatively low cost. However, three-dimensional dynamic response monitoring and measurement on large-scale structures pose challenges of camera calibration and synchronization to image analysis. Without satisfactory camera position and orientation obtained from calibration and well-synchronized imaging, significant errors would occur in the dynamic responses during image analysis and stereo triangulation. This paper introduces two camera calibration approaches that are suitable for large-scale structural experiments, as well as a synchronization method to estimate the time difference between two cameras and further minimize the error of stereo triangulation. Two structural experiments are used to verify the calibration approaches and the synchronization method to acquire dynamic responses. The results demonstrate the performance and accuracy improvement by using the proposed methods.

Metrological characterization of 3D biometric face recognition systems in actual operating conditions

<p>Nowadays, face recognition systems are going to widespread in many fields of application, from automatic user login for financial activities and access to restricted areas, to surveillance for improving security in airports and railway stations, to cite a few.<br />In such scenarios, the architectures based on stereo vision and 3D reconstruction of the face are going to assume a predominant role because they can generally assure a better reliability than solutions based on a single camera (which make use of a single image instead of a couple of images). To realize such systems, different architectures can be considered by varying the positioning of the pair of cameras with respect to the face of the subject to be identified, as well as both kind and resolution of camera considered. These parameters can affect the correct decision rate of the system in classifying the input face, especially in presence of image uncertainty.<br />In this paper, several 3D architectures differing in camera specifications and geometrical positioning of the camera pair (with respect to the input face) are realized and compared. The detection of facial features in the images is made by adopting a popular method based on the Active Appearance Model (AAM) algorithm. 3D position of facial features is then obtained by means of stereo triangulation. The performance of the realized systems has been compared in terms of sensitivity to the quantities of influence and related uncertainty, and of typical indexes for the analysis of classification systems. Main results of such comparison show that the best performance can be reached by reducing the distance between cameras and subject to be identified and by minimizing the horizontal angle between the plane containing the camera pair axis and the face to be identified.</p>

Test Setup for Determination of Bat Flight Kinematics

Bats can achieve great feats of agility by utilizing more than twenty degrees of freedom, however, this makes aerodynamic data extremely difficult to obtain, even for the simplest flight modes. Therefore, a test setup was developed to accurately map bat flight kinematics throughout the entire flap cycle with minimal occlusion using 3-D Stereo Triangulation. A controlled flight room was constructed in which the two bat species, Hipposideros Pratti and Hipposideros Armiger, could be safely monitored and directed. 3-D Stereo Triangulation was performed using three rings of ten high-speed cameras, as well as matte white markers on all of the major joint locations and scattered throughout the wing membrane. Flight direction was controlled by use of an elevated flight tunnel, ensuring that each bat would fly straight through all three camera rings. Obstacles were also incorporated to investigate the incredible banking and flight reversal that these bats are capable of. This research builds off of a previous study where markers were only placed on the major joint locations and Two-Camera Stereo Triangulation was used.

Stereo Model Selection and Point Cloud Filtering using an Out-of-Core Octree

Dense image matching methods enable the retrieval of dense surface information using any kind of imagery. The best quality can be achieved for highly overlapping datasets, which avoids occlusions and provides highly redundant observations. Thus, images are acquired close to each other. This leads to datasets with increasing size &ndash; especially when large scenes are captured. While image acquisition can be performed in relatively short time, more time is required for data processing due to the computational complexity of the involved algorithms. For the dense surface reconstruction task, <i>Multi-View Stereo</i> algorithms can be used – which are typically beneficial due to the efficiency of image matching on stereo models. Our dense image matching solution <i>SURE</i> uses such an approach, where the result of stereo matching is fused using a multi-stereo triangulation in order to exploit the available redundancy. One key challenge of such <i>Multi-View Stereo</i> methods is the selection of suitable stereo models, where object space information should be considered to avoid unnecessary processing. Subsequently, the dense image matching step provides up to one 3D point for each pixel, which leads to massive point clouds. This large amount of 3D data needs to be filtered and integrated efficiently in object space. Within this paper, we present an <i>out-of-core octree</i>, which enables neighborhood and overlap analysis between point clouds. It is used on low-resolution point clouds to support the stereo model selection. Also, this tree is designed for the processing of massive point clouds with low memory requirements and thus can be used to perform outlier rejection, redundancy removal and resampling.

A Methodology for the Kinematic and Unsteady Dynamics Analysis of Bat Flight

A methodology to capture and post-process bat flight 3-D Stereo Triangulation data to formulate an approximated rigid body kinematic model was investigated. Bat flight is unique in nature due to the bats inherent agility and many degrees of freedom when compared to other flying animals. This complexity makes capturing accurate aerodynamic data very difficult. Unlike insects, which utilize few degrees of freedom and a high flap frequency for sustained flight and maneuverability, the agility of bats comes in part from the many degrees of freedom present in the bat wing. In order to better understand the aerodynamics present in bat flight, bats Hipposideridae (Old World leaf-nosed bats) were examined. The trajectories of critical points along the bat wings were recorded using 3D stereo triangulation techniques to capture the complexities of the bat flight. Markers were placed at all the joint locations along the bat wing. The resulting trajectories were then translated into a periodic kinematic model for future computational use.