scholarly journals A Bridge Neural Network-Based Optical-SAR Image Joint Intelligent Interpretation Framework

2021 ◽  
Vol 2021 ◽  
pp. 1-10
Author(s):  
Meiyu Huang ◽  
Yao Xu ◽  
Lixin Qian ◽  
Weili Shi ◽  
Yaqin Zhang ◽  
...  
Keyword(s):  
Neural Network ◽  
Feature Space ◽  
Representation Learning ◽  
Learning Ability ◽  
Sar Image ◽  
Sar Images ◽  
Optical Images ◽  
Image Patches ◽  
Pair Matching ◽  
Correlated Information

The current interpretation technology of remote sensing images is mainly focused on single-modal data, which cannot fully utilize the complementary and correlated information of multimodal data with heterogeneous characteristics, especially for synthetic aperture radar (SAR) data and optical imagery. To solve this problem, we propose a bridge neural network- (BNN-) based optical-SAR image joint intelligent interpretation framework, optimizing the feature correlation between optical and SAR images through optical-SAR matching tasks. It adopts BNN to effectively improve the capability of common feature extraction of optical and SAR images and thus improving the accuracy and application scenarios of specific intelligent interpretation tasks for optical-SAR/SAR/optical images. Specifically, BNN projects optical and SAR images into a common feature space and mines their correlation through pair matching. Further, to deeply exploit the correlation between optical and SAR images and ensure the great representation learning ability of BNN, we build the QXS-SAROPT dataset containing 20,000 pairs of perfectly aligned optical-SAR image patches with diverse scenes of high resolutions. Experimental results on optical-to-SAR crossmodal object detection demonstrate the effectiveness and superiority of our framework. In particular, based on the QXS-SAROPT dataset, our framework can achieve up to 96% high accuracy on four benchmark SAR ship detection datasets.

scholarly journals Task-Driven Common Representation Learning via Bridge Neural Network

2019 ◽  
Vol 33 ◽  
pp. 5573-5580
Author(s):  
Yao Xu ◽  
Xueshuang Xiang ◽  
Meiyu Huang
Keyword(s):  
Neural Network ◽  
Canonical Correlation ◽  
State Of The Art ◽  
Feature Space ◽  
Representation Learning ◽  
Data Sources ◽  
Common Representation ◽  
Total Correlation ◽  
Pair Matching ◽  
Training Objective

This paper introduces a novel deep learning based method, named bridge neural network (BNN) to dig the potential relationship between two given data sources task by task. The proposed approach employs two convolutional neural networks that project the two data sources into a feature space to learn the desired common representation required by the specific task. The training objective with artificial negative samples is introduced with the ability of mini-batch training and it’s asymptotically equivalent to maximizing the total correlation of the two data sources, which is verified by the theoretical analysis. The experiments on the tasks, including pair matching, canonical correlation analysis, transfer learning, and reconstruction demonstrate the state-of-the-art performance of BNN, which may provide new insights into the aspect of common representation learning.

scholarly journals Rethinking the Random Cropping Data Augmentation Method Used in the Training of CNN-Based SAR Image Ship Detector

2020 ◽  
Vol 13 (1) ◽  
pp. 34
Author(s):  
Rong Yang ◽  
Robert Wang ◽  
Yunkai Deng ◽  
Xiaoxue Jia ◽  
Heng Zhang
Keyword(s):  
Neural Network ◽  
Data Augmentation ◽  
Back Propagation ◽  
Detection Performance ◽  
Training Data ◽  
Sar Image ◽  
Optical Images ◽  
The Neural Network ◽  
Effective Training ◽  
Standard Configuration

The random cropping data augmentation method is widely used to train convolutional neural network (CNN)-based target detectors to detect targets in optical images (e.g., COCO datasets). It can expand the scale of the dataset dozens of times while consuming only a small amount of calculations when training the neural network detector. In addition, random cropping can also greatly enhance the spatial robustness of the model, because it can make the same target appear in different positions of the sample image. Nowadays, random cropping and random flipping have become the standard configuration for those tasks with limited training data, which makes it natural to introduce them into the training of CNN-based synthetic aperture radar (SAR) image ship detectors. However, in this paper, we show that the introduction of traditional random cropping methods directly in the training of the CNN-based SAR image ship detector may generate a lot of noise in the gradient during back propagation, which hurts the detection performance. In order to eliminate the noise in the training gradient, a simple and effective training method based on feature map mask is proposed. Experiments prove that the proposed method can effectively eliminate the gradient noise introduced by random cropping and significantly improve the detection performance under a variety of evaluation indicators without increasing inference cost.

scholarly journals High-Resolution SAR Image Classification Using Multi-Scale Deep Feature Fusion and Covariance Pooling Manifold Network

2021 ◽  
Vol 13 (2) ◽  
pp. 328
Author(s):  
Wenkai Liang ◽  
Yan Wu ◽  
Ming Li ◽  
Yice Cao ◽  
Xin Hu
Keyword(s):  
High Resolution ◽  
Image Classification ◽  
Feature Fusion ◽  
Representation Learning ◽  
Sar Image ◽  
Gabor Filtering ◽  
Feature Maps ◽  
Sar Images ◽  
Multi Scale ◽  
Deep Feature

The classification of high-resolution (HR) synthetic aperture radar (SAR) images is of great importance for SAR scene interpretation and application. However, the presence of intricate spatial structural patterns and complex statistical nature makes SAR image classification a challenging task, especially in the case of limited labeled SAR data. This paper proposes a novel HR SAR image classification method, using a multi-scale deep feature fusion network and covariance pooling manifold network (MFFN-CPMN). MFFN-CPMN combines the advantages of local spatial features and global statistical properties and considers the multi-feature information fusion of SAR images in representation learning. First, we propose a Gabor-filtering-based multi-scale feature fusion network (MFFN) to capture the spatial pattern and get the discriminative features of SAR images. The MFFN belongs to a deep convolutional neural network (CNN). To make full use of a large amount of unlabeled data, the weights of each layer of MFFN are optimized by unsupervised denoising dual-sparse encoder. Moreover, the feature fusion strategy in MFFN can effectively exploit the complementary information between different levels and different scales. Second, we utilize a covariance pooling manifold network to extract further the global second-order statistics of SAR images over the fusional feature maps. Finally, the obtained covariance descriptor is more distinct for various land covers. Experimental results on four HR SAR images demonstrate the effectiveness of the proposed method and achieve promising results over other related algorithms.

scholarly journals Change Detection Based on Multi-Grained Cascade Forest and Multi-Scale Fusion for SAR Images

2019 ◽  
Vol 11 (2) ◽  
pp. 142 ◽  
Author(s):  
Wenping Ma ◽  
Hui Yang ◽  
Yue Wu ◽  
Yunta Xiong ◽  
Tao Hu ◽  
...  
Keyword(s):  
Change Detection ◽  
Representation Learning ◽  
Local Information ◽  
Learning Ability ◽  
Detection Methods ◽  
Detection Accuracy ◽  
Image Block ◽  
Sar Images ◽  
Difference Image ◽  
Multi Scale

In this paper, a novel change detection approach based on multi-grained cascade forest(gcForest) and multi-scale fusion for synthetic aperture radar (SAR) images is proposed. It detectsthe changed and unchanged areas of the images by using the well-trained gcForest. Most existingchange detection methods need to select the appropriate size of the image block. However, thesingle size image block only provides a part of the local information, and gcForest cannot achieve agood effect on the image representation learning ability. Therefore, the proposed approach choosesdifferent sizes of image blocks as the input of gcForest, which can learn more image characteristicsand reduce the influence of the local information of the image on the classification result as well.In addition, in order to improve the detection accuracy of those pixels whose gray value changesabruptly, the proposed approach combines gradient information of the difference image with theprobability map obtained from the well-trained gcForest. Therefore, the image edge information canbe enhanced and the accuracy of edge detection can be improved by extracting the image gradientinformation. Experiments on four data sets indicate that the proposed approach outperforms otherstate-of-the-art algorithms.

Speckle Noise Filtering Using Back-Propagation Multi-Layer Perceptron Network in Synthetic Aperture Radar Image

2020 ◽  
pp. 489-510
Author(s):  
Khwairakpam Amitab ◽  
Debdatta Kandar ◽  
Arnab K. Maji
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Synthetic Aperture Radar ◽  
Back Propagation ◽  
Speckle Noise ◽  
Synthetic Aperture ◽  
Sar Image ◽  
Multi Layer Perceptron ◽  
Sar Images ◽  
Artificial Neural

Synthetic Aperture Radar (SAR) are imaging Radar, it uses electromagnetic radiation to illuminate the scanned surface and produce high resolution images in all-weather condition, day and night. Interference of signals causes noise and degrades the quality of the image, it causes serious difficulty in analyzing the images. Speckle is multiplicative noise that inherently exist in SAR images. Artificial Neural Network (ANN) have the capability of learning and is gaining popularity in SAR image processing. Multi-Layer Perceptron (MLP) is a feed forward artificial neural network model that consists of an input layer, several hidden layers, and an output layer. We have simulated MLP with two hidden layer in Matlab. Speckle noises were added to the target SAR image and applied MLP for speckle noise reduction. It is found that speckle noise in SAR images can be reduced by using MLP. We have considered Log-sigmoid, Tan-Sigmoid and Linear Transfer Function for the hidden layers. The MLP network are trained using Gradient descent with momentum back propagation, Resilient back propagation and Levenberg-Marquardt back propagation and comparatively evaluated the performance.

scholarly journals Sentinel-2 Cloud Removal Considering Ground Changes by Fusing Multitemporal SAR and Optical Images

2021 ◽  
Vol 13 (19) ◽  
pp. 3998
Author(s):  
Jianhao Gao ◽  
Yang Yi ◽  
Tang Wei ◽  
Haoguan Zhang
Keyword(s):  
Neural Network ◽  
Real Time ◽  
Sar Images ◽  
The Real ◽  
Removal Process ◽  
Optical Images ◽  
Cloud Removal ◽  
Real Time Information ◽  
Time Information ◽  
Sentinel 2

Publicly available optical remote sensing images from platforms such as Sentinel-2 satellites contribute much to the Earth observation and research tasks. However, information loss caused by clouds largely decreases the availability of usable optical images so reconstructing the missing information is important. Existing reconstruction methods can hardly reflect the real-time information because they mainly make use of multitemporal optical images as reference. To capture the real-time information in the cloud removal process, Synthetic Aperture Radar (SAR) images can serve as the reference images due to the cloud penetrability of SAR imaging. Nevertheless, large datasets are necessary because existing SAR-based cloud removal methods depend on network training. In this paper, we integrate the merits of multitemporal optical images and SAR images to the cloud removal process, the results of which can reflect the ground information change, in a simple convolution neural network. Although the proposed method is based on deep neural network, it can directly operate on the target image without training datasets. We conduct several simulation and real data experiments of cloud removal in Sentinel-2 images with multitemporal Sentinel-1 SAR images and Sentinel-2 optical images. Experiment results show that the proposed method outperforms those state-of-the-art multitemporal-based methods and overcomes the constraint of datasets of those SAR-based methods.

scholarly journals A Fusion Method of Optical Image and SAR Image Based on Dense-UGAN and Gram–Schmidt Transformation

2021 ◽  
Vol 13 (21) ◽  
pp. 4274
Author(s):  
Yingying Kong ◽  
Fang Hong ◽  
Henry Leung ◽  
Xiangyang Peng
Keyword(s):  
Loss Function ◽  
Texture Features ◽  
Speckle Noise ◽  
Source Image ◽  
Fusion Method ◽  
Sar Image ◽  
Sar Images ◽  
Spatial Features ◽  
Optical Images ◽  
Fusion Methods

To solve the problems such as obvious speckle noise and serious spectral distortion when existing fusion methods are applied to the fusion of optical and SAR images, this paper proposes a fusion method for optical and SAR images based on Dense-UGAN and Gram–Schmidt transformation. Firstly, dense connection with U-shaped network (Dense-UGAN) are used in GAN generator to deepen the network structure and obtain deeper source image information. Secondly, according to the particularity of SAR imaging mechanism, SGLCM loss for preserving SAR texture features and PSNR loss for reducing SAR speckle noise are introduced into the generator loss function. Meanwhile in order to keep more SAR image structure, SSIM loss is introduced to discriminator loss function to make the generated image retain more spatial features. In this way, the generated high-resolution image has both optical contour characteristics and SAR texture characteristics. Finally, the GS transformation of optical and generated image retains the necessary spectral properties. Experimental results show that the proposed method can well preserve the spectral information of optical images and texture information of SAR images, and also reduce the generation of speckle noise at the same time. The metrics are superior to other algorithms that currently perform well.

scholarly journals BUILDING DETECTION FROM SAR IMAGES USING UNET DEEP LEARNING METHOD

2020 ◽  
Vol XLIV-4/W3-2020 ◽  
pp. 215-218
Author(s):  
R. A. Emek ◽  
N. Demir
Keyword(s):  
Deep Learning ◽  
Object Detection ◽  
Optical Image ◽  
Building Detection ◽  
Multispectral Images ◽  
Learning Method ◽  
Sar Images ◽  
Optical Images ◽  
Image Patches ◽  
Sentinel 2

Abstract. SAR images are different from the optical images in terms of image properties with the values of scattering instead of reflectance. This makes SAR images difficult to apply the traditional object detection methodologies. In recent years, deep learning models are frequently used in segmentation and object detection purposes. In this study, we have investigated the potential of U-Net models for building detection from SAR and optical image fusion. The datasets used are Sentinel 1 SAR and Sentinel-2 multispectral images, provided from ‘SpaceNet 6 Multi Sensor All-Weather Mapping’ challenge. These images cover an area of 120 km2 in Rotterdam, the Netherlands. As training datasets 20 pieces of 900 by 900 pixel sized HV polarized and optical image patches have been used together. The calculated loss value is 0.4 and the accuracy is 81%.

Speckle Noise Filtering Using Back-Propagation Multi-Layer Perceptron Network in Synthetic Aperture Radar Image

2016 ◽  
pp. 280-301
Author(s):  
Khwairakpam Amitab ◽  
Debdatta Kandar ◽  
Arnab K. Maji
Keyword(s):  
Neural Network ◽  
Artificial Neural Network ◽  
Synthetic Aperture Radar ◽  
Back Propagation ◽  
Speckle Noise ◽  
Synthetic Aperture ◽  
Sar Image ◽  
Multi Layer Perceptron ◽  
Sar Images ◽  
Artificial Neural

Synthetic Aperture Radar (SAR) are imaging Radar, it uses electromagnetic radiation to illuminate the scanned surface and produce high resolution images in all-weather condition, day and night. Interference of signals causes noise and degrades the quality of the image, it causes serious difficulty in analyzing the images. Speckle is multiplicative noise that inherently exist in SAR images. Artificial Neural Network (ANN) have the capability of learning and is gaining popularity in SAR image processing. Multi-Layer Perceptron (MLP) is a feed forward artificial neural network model that consists of an input layer, several hidden layers, and an output layer. We have simulated MLP with two hidden layer in Matlab. Speckle noises were added to the target SAR image and applied MLP for speckle noise reduction. It is found that speckle noise in SAR images can be reduced by using MLP. We have considered Log-sigmoid, Tan-Sigmoid and Linear Transfer Function for the hidden layers. The MLP network are trained using Gradient descent with momentum back propagation, Resilient back propagation and Levenberg-Marquardt back propagation and comparatively evaluated the performance.

scholarly journals Siamese CNN-BiLSTM Architecture for 3D Shape Representation Learning

2018 ◽  
Author(s):  
Guoxian Dai ◽  
Jin Xie ◽  
Yi Fang
Keyword(s):  
Neural Network ◽  
Loss Function ◽  
Short Term Memory ◽  
Shape Representation ◽  
Feature Space ◽  
Representation Learning ◽  
3D Shape ◽  
Aggregate Information ◽  
3D Shapes ◽  
2D Images

Learning a 3D shape representation from a collection of its rendered 2D images has been extensively studied. However, existing view-based techniques have not yet fully exploited the information among all the views of projections. In this paper, by employing recurrent neural network to efficiently capture features across different views, we propose a siamese CNN-BiLSTM network for 3D shape representation learning. The proposed method minimizes a discriminative loss function to learn a deep nonlinear transformation, mapping 3D shapes from the original space into a nonlinear feature space. In the transformed space, the distance of 3D shapes with the same label is minimized, otherwise the distance is maximized to a large margin. Specifically, the 3D shapes are first projected into a group of 2D images from different views. Then convolutional neural network (CNN) is adopted to extract features from different view images, followed by a bidirectional long short-term memory (LSTM) to aggregate information across different views. Finally, we construct the whole CNN-BiLSTM network into a siamese structure with contrastive loss function. Our proposed method is evaluated on two benchmarks, ModelNet40 and SHREC 2014, demonstrating superiority over the state-of-the-art methods.

Sign in / Sign up

Export Citation Format

Share Document