Landscape Similarity Analysis Using Texture Encoded Deep-Learning Features on Unclassified Remote Sensing Imagery

Convolutional neural networks (CNNs) are known for their ability to learn shape and texture descriptors useful for object detection, pattern recognition, and classification problems. Deeper layer filters of CNN generally learn global image information vital for whole-scene or object discrimination. In landscape pattern comparison, however, dense localized information encoded in shallow layers can contain discriminative information for characterizing changes across image local regions but are often lost in the deeper and non-spatial fully connected layers. Such localized features hold potential for identifying, as well as characterizing, process–pattern change across space and time. In this paper, we propose a simple yet effective texture-based CNN (Tex-CNN) via a feature concatenation framework which results in capturing and learning texture descriptors. The traditional CNN architecture was adopted as a baseline for assessing the performance of Tex-CNN. We utilized 75% and 25% of the image data for model training and validation, respectively. To test the models’ generalization, we used a separate set of imagery from the Aerial Imagery Dataset (AID) and Sentinel-2 for model development and independent validation. The classical CNN and the Tex-CNN classification accuracies in the AID were 91.67% and 96.33%, respectively. Tex-CNN accuracy was either on par with or outcompeted state-of-the-art methods. Independent validation on Sentinel-2 data had good performance for most scene types but had difficulty discriminating farm scenes, likely due to geometric generalization of discriminative features at the coarser scale. In both datasets, the Tex-CNN outperformed the classical CNN architecture. Using the Tex-CNN, gradient-based spatial attention maps (feature maps) which contain discriminative pattern information are extracted and subsequently employed for mapping landscape similarity. To enhance the discriminative capacity of the feature maps, we further perform spatial filtering, using PCA and select eigen maps with the top eigen values. We show that CNN feature maps provide descriptors capable of characterizing and quantifying landscape (dis)similarity. Using the feature maps histogram of oriented gradient vectors and computing their Earth Movers Distances, our method effectively identified similar landscape types with over 60% of target-reference scene comparisons showing smaller Earth Movers Distance (EMD) (e.g., 0.01), while different landscape types tended to show large EMD (e.g., 0.05) in the benchmark AID. We hope this proposal will inspire further research into the use of CNN layer feature maps in landscape similarity assessment, as well as in change detection.

Measuring Musical Rhythm Similarity: Statistical Features Versus Transformation Methods

Two approaches to measuring the similarity between symbolically notated musical rhythms are compared with each other and with human judgments of perceived similarity. The first is the edit-distance, a popular transformation method, applied to the symbolic rhythm sequences. The second approach employs the histograms of the inter-onset-intervals (IOIs) calculated from the rhythms. Furthermore, two methods for dealing with the histograms are also compared. The first utilizes the Mallows distance, a transformation method akin to the Earth-Movers distance popular in computer vision, and the second extracts a group of standard statistical features, used in music information retrieval, from the IOI-histograms. The measures are compared using four contrastive musical rhythm data sets by means of statistical Mantel tests that compute correlation coefficients between the various dissimilarity matrices. The results provide evidence from the aural domain, that transformation methods such as the edit distance are superior to feature-based methods for predicting human judgments of similarity. The evidence also supports the hypothesis that IOI-histogram-based methods are better than music-theoretical structural features computed from the rhythms themselves, provided that the rhythms do not share identical IOI histograms.

A FAST APPROXIMATION OF THE EARTH-MOVERS DISTANCE BETWEEN MULTIDIMENSIONAL HISTOGRAMS

We present an efficient algorithm for computing a sub-optimal Earth Movers' Distance (EMD) between multidimensional histograms called EMD- g f, which is not limited to any type of measurement. Some algorithms that find a cross-bin distance between histograms have been proposed in the literature. Nevertheless, most of this research has been applied on 1D-histograms or on nD-histograms but with limited types of measurements. The EMD is a cross-bin distance between nD-histograms with any ground distance. Experimental validation shows that it obtains good retrieval results although the main drawback of this method is its cubic computational cost, O(z3), z being the total number of bins. The worst-case complexity of EMD- g f is O(z2), although the obtained average computational cost in the experiments is near O(m2), where m represents the number of bins per dimension, which is clearly lower than the computational cost of the EMD algorithm. Moreover, the experiments using real data show similar retrieval results.