scholarly journals TextRS: Deep Bidirectional Triplet Network for Matching Text to Remote Sensing Images

2020 ◽  
Vol 12 (3) ◽  
pp. 405 ◽  
Author(s):  
Taghreed Abdullah ◽  
Yakoub Bazi ◽  
Mohamad M. Al Rahhal ◽  
Mohamed L. Mekhalfi ◽  
Lalitha Rangarajan ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Image Matching ◽  
Short Term Memory ◽  
Remote Sensing Image ◽  
Remote Sensing Images ◽  
Short Term ◽  
Fusion Strategy ◽  
Memory Network ◽  
Long Short Term Memory ◽  
Image Representations

Exploring the relevance between images and their respective natural language descriptions, due to its paramount importance, is regarded as the next frontier in the general computer vision literature. Thus, recently several works have attempted to map visual attributes onto their corresponding textual tenor with certain success. However, this line of research has not been widespread in the remote sensing community. On this point, our contribution is three-pronged. First, we construct a new dataset for text-image matching tasks, termed TextRS, by collecting images from four well-known different scene datasets, namely AID, Merced, PatternNet, and NWPU datasets. Each image is annotated by five different sentences. All the five sentences were allocated by five people to evidence the diversity. Second, we put forth a novel Deep Bidirectional Triplet Network (DBTN) for text to image matching. Unlike traditional remote sensing image-to-image retrieval, our paradigm seeks to carry out the retrieval by matching text to image representations. To achieve that, we propose to learn a bidirectional triplet network, which is composed of Long Short Term Memory network (LSTM) and pre-trained Convolutional Neural Networks (CNNs) based on (EfficientNet-B2, ResNet-50, Inception-v3, and VGG16). Third, we top the proposed architecture with an average fusion strategy to fuse the features pertaining to the five image sentences, which enables learning of more robust embedding. The performances of the method expressed in terms Recall@K representing the presence of the relevant image among the top K retrieved images to the query text shows promising results as it yields 17.20%, 51.39%, and 73.02% for K = 1, 5, and 10, respectively.

scholarly journals Semi-Supervised Convolutional Long Short-Term Memory Neural Networks for Time Series Land Cover Classification

2021 ◽  
Vol 13 (17) ◽  
pp. 3504
Author(s):  
Jing Shen ◽  
Chao Tao ◽  
Ji Qi ◽  
Hao Wang
Keyword(s):  
Remote Sensing ◽  
Neural Networks ◽  
Time Series ◽  
Land Cover ◽  
Classification Accuracy ◽  
Short Term Memory ◽  
Remote Sensing Images ◽  
Short Term ◽  
Term Memory ◽  
Long Short Term Memory

Time series images with temporal features are beneficial to improve the classification accuracy. For abstract temporal and spatial contextual information, deep neural networks have become an effective method. However, there is usually a lack of sufficient samples in network training: one is the loss of images or the discontinuous distribution of time series data because of the inevitable cloud cover, and the other is the lack of known labeled data. In this paper, we proposed a Semi-supervised convolutional Long Short-Term Memory neural network (SemiLSTM) for time series remote sensing images, which was validated on three data sets with different time distributions. It achieves an accurate and automated land cover classification via a small number of labeled samples and a large number of unlabeled samples. Besides, it is a robust classification algorithm for time series optical images with cloud coverage, which reduces the requirements for cloudless remote sensing images and can be widely used in areas that are often obscured by clouds, such as subtropical areas. In conclusion, this method makes full advantage of spectral-spatial-temporal characteristics under the condition of limited training samples, especially expanding time context information to enhance classification accuracy.

scholarly journals Small Floating Target Detection Method Based on Chaotic Long Short-Term Memory Network

2021 ◽  
Vol 9 (6) ◽  
pp. 651
Author(s):  
Yan Yan ◽  
Hongyan Xing
Keyword(s):  
High Frequency ◽  
Short Term Memory ◽  
Detection Method ◽  
Low Frequency ◽  
Sea Clutter ◽  
Short Term ◽  
Term Memory ◽  
Memory Network ◽  
Long Short Term Memory ◽  
Small Targets

In order for the detection ability of floating small targets in sea clutter to be improved, on the basis of the complete ensemble empirical mode decomposition (CEEMD) algorithm, the high-frequency parts and low-frequency parts are determined by the energy proportion of the intrinsic mode function (IMF); the high-frequency part is denoised by wavelet packet transform (WPT), whereas the denoised high-frequency IMFs and low-frequency IMFs reconstruct the pure sea clutter signal together. According to the chaotic characteristics of sea clutter, we proposed an adaptive training timesteps strategy. The training timesteps of network were determined by the width of embedded window, and the chaotic long short-term memory network detection was designed. The sea clutter signals after denoising were predicted by chaotic long short-term memory (LSTM) network, and small target signals were detected from the prediction errors. The experimental results showed that the CEEMD-WPT algorithm was consistent with the target distribution characteristics of sea clutter, and the denoising performance was improved by 33.6% on average. The proposed chaotic long- and short-term memory network, which determines the training step length according to the width of embedded window, is a new detection method that can accurately detect small targets submerged in the background of sea clutter.

Sign in / Sign up

Export Citation Format

Share Document