A Faster and More Effective Cross-View Matching Method of UAV and Satellite Images for UAV Geolocalization

Cross-view geolocalization matches the same target in different images from various views, such as views of unmanned aerial vehicles (UAVs) and satellites, which is a key technology for UAVs to autonomously locate and navigate without a positioning system (e.g., GPS and GNSS). The most challenging aspect in this area is the shifting of targets and nonuniform scales among different views. Published methods focus on extracting coarse features from parts of images, but neglect the relationship between different views, and the influence of scale and shifting. To bridge this gap, an effective network is proposed with well-designed structures, referred to as multiscale block attention (MSBA), based on a local pattern network. MSBA cuts images into several parts with different scales, among which self-attention is applied to make feature extraction more efficient. The features of different views are extracted by a multibranch structure, which was designed to make different branches learn from each other, leading to a more subtle relationship between views. The method was implemented with the newest UAV-based geolocalization dataset. Compared with the existing state-of-the-art (SOTA) method, MSBA accuracy improved by almost 10% when the inference time was equal to that of the SOTA method; when the accuracy of MSBA was the same as that of the SOTA method, inference time was shortened by 30%.

A discovery about the positional distribution pattern among candidate homologous pixels and its potential application in aerial multi-view image matching

In aerial multi-view photogrammetry, whether there is a special positional distribution pattern among candidate homologous pixels of a matching pixel in the multi-view images? If so, can this positional pattern be used to precisely confirm the real homologous pixels? These problems have not been studied at present. Therefore, the study of the positional distribution pattern among candidate homologous pixels based on the adjustment theory in surveying is investigated in this paper. Firstly, the definition and computing method of pixel’s pseudo object-space coordinates are given, which can transform the problem of multi-view matching for confirming real homologous pixels into the problem of surveying adjustment for computing the pseudo object-space coordinates of the matching pixel. Secondly, according to the surveying adjustment theory, the standardized residual of each candidate homologous pixel of the matching pixel is figured out, and the positional distribution pattern among these candidate pixels is theoretically inferred utilizing the quantitative index of standardized residual. Lastly, actual aerial images acquired by different sensors are used to carry out experimental verification of the theoretical inference. Experimental results prove not only that there is a specific positional distribution pattern among candidate homologous pixels, but also that this positional distribution pattern can be used to develop a new object-side multi-view image matching method. The proposed study has an important reference value on resolving the defects of existing image-side multi-view matching methods at the mechanism level.

Double-View Matching Network for Few-Shot Learning to Classify Covid-19 in X-ray images

The research topic presented in this paper belongs to small training data problem in machine learning (especially in deep learning), it intends to help the work of those working in medicine by analyzing pathological X-ray recordings, using only very few images. This scenario is a particularly hot issue nowadays: how could a new disease for which only limited data are available be diagnosed using features of previous diseases? In this problem, so-called few-shot learning, the difficulty of the classification task is to learn the unique feature characteristics associated with the classes. Although there are solutions, but if the images come from different views, they will not handle these views well. We proposed an improved method, so-called Double-View Matching Network (DVMN based on the deep neural network), which solves the few-shot learning problem as well as the different views of the pathological recordings in the images. The main contribution of this is the convolutional neural network for feature extraction and handling the multi-view in image representation. Our method was tested in the classification of images showing unknown COVID-19 symptoms in an environment designed for learning a few samples, with prior meta-learning on images of other diseases only. The results show that DVMN reaches better accuracy on multi-view dataset than simple Matching Network without multi-view handling.

3D hypothesis clustering for cross-view matching in multi-person motion capture

We present a multiview method for markerless motion capture of multiple people. The main challenge in this problem is to determine cross-view correspondences for the 2D joints in the presence of noise. We propose a 3D hypothesis clustering technique to solve this problem. The core idea is to transform joint matching in 2D space into a clustering problem in a 3D hypothesis space. In this way, evidence from photometric appearance, multiview geometry, and bone length can be integrated to solve the clustering problem efficiently and robustly. Each cluster encodes a set of matched 2D joints for the same person across different views, from which the 3D joints can be effectively inferred. We then assemble the inferred 3D joints to form full-body skeletons for all persons in a bottom–up way. Our experiments demonstrate the robustness of our approach even in challenging cases with heavy occlusion, closely interacting people, and few cameras. We have evaluated our method on many datasets, and our results show that it has significantly lower estimation errors than many state-of-the-art methods.