scholarly journals Hybrid Attention Based Residual Network for Pansharpening

2021 ◽  
Vol 13 (10) ◽  
pp. 1962
Author(s):  
Qin Liu ◽  
Letong Han ◽  
Rui Tan ◽  
Hongfei Fan ◽  
Weiqi Li ◽  
...  
Keyword(s):  
Neural Network ◽  
State Of The Art ◽  
Object Identification ◽  
Spectral Information ◽  
Spectral Distortion ◽  
Residual Network ◽  
Spatial Features ◽  
Fused Image ◽  
Ground Object ◽  
The Rich

Pansharpening aims at fusing the rich spectral information of multispectral(MS) images and the spatial details of panchromatic(PAN) images to generate a fused image with both high resolutions. In general, the existing pansharpening methods suffer from the problems of spectral distortion and lack of spatial detail information, which might prevent the accuracy computation for ground object identification. To alleviate these problems, we propose a Hybrid Attention mechanism-based Residual Neural Network(HARNN) . In the proposed network, we develop an encoder attention module in the feature extraction part to better utilize the spectral and spatial features of MS and PAN images. Furthermore, the fusion attention module is designed to alleviate spectral distortion and improve contour details of the fused image. A series of ablation and contrast experiments are conducted on GF-1 and GF-2 datasets. The fusion results with less distorted pixels and more spatial details demonstrate that HARNN can implement the pansharpening task effectively, which outperforms the state-of-the-art algorithms.

scholarly journals Self-Paced Convolutional Neural Network for PolSAR Images Classification

2019 ◽  
Vol 11 (4) ◽  
pp. 424 ◽  
Author(s):  
Changzhe Jiao ◽  
Xinlin Wang ◽  
Shuiping Gou ◽  
Wenshuai Chen ◽  
Debo Li ◽  
...  
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Synthetic Aperture ◽  
Training Process ◽  
Polarimetric Synthetic Aperture Radar ◽  
Learning Mechanism ◽  
3 Dimensional ◽  
Spatial Features ◽  
Rgb Values

Fully polarimetric synthetic aperture radar (PolSAR) can transmit and receive electromagnetic energy on four polarization channels (HH, HV, VH, VV). The data acquired from four channels have both similarities and complementarities. Utilizing the information between the four channels can considerably improve the performance of PolSAR image classification. Convolutional neural network can be used to extract the channel-spatial features of PolSAR images. Self-paced learning has been demonstrated to be instrumental in enhancing the learning robustness of convolutional neural network. In this paper, a novel classification method for PolSAR images using self-paced convolutional neural network (SPCNN) is proposed. In our method, each pixel is denoted by a 3-dimensional tensor block formed by its scattering intensity values on four channels, Pauli’s RGB values and its neighborhood information. Then, we train SPCNN to extract the channel-spatial features and obtain the classification results. Inspired by self-paced learning, SPCNN learns the easier samples first and gradually involves more difficult samples into the training process. This learning mechanism can make network converge to better values. The proposed method achieved state-of-the-art performances on four real PolSAR dataset.

scholarly journals DRHNet: A Deep Residual Network Based on Heterogeneous Kernel for Steganalysis

2020 ◽  
Vol 2020 ◽  
pp. 1-9
Author(s):  
Yang Xu ◽  
Zixi Fu ◽  
Guiyong Xu ◽  
Sicong Zhang ◽  
Xiaoyao Xie
Keyword(s):  
Neural Networks ◽  
Heterogeneous Network ◽  
State Of The Art ◽  
Training Phase ◽  
Image Size ◽  
Residual Network ◽  
Training Time ◽  
Learning Framework ◽  
Residual Learning ◽  
The Rich

Convolutional neural networks as steganalysis have problems such as poor versatility, long training time, and limited image size. For these problems, we present a heterogeneous kernel residual learning framework called DRHNet—Dual Residual Heterogeneous Network—to save time on the networks during the training phase. Instead of using the image as an input of the network, we extract and merge the images into a feature matrix using the rich model and use the generated feature matrix as the real input of the network. The architecture we proposed has good versatility and can reduce the computation and the number of parameters while still getting higher accuracy. On BOSSbase 1.01, we evaluate the performance of DRHNet in the setting of the spatial domain and frequency domain. The preliminary experimental results show that DRHNet shows excellent steganalysis performance against the state-of-the-art steganographic algorithms.

scholarly journals DR-Net: An Improved Network for Building Extraction from High Resolution Remote Sensing Image

2021 ◽  
Vol 13 (2) ◽  
pp. 294
Author(s):  
Meng Chen ◽  
Jianjun Wu ◽  
Leizhen Liu ◽  
Wenhui Zhao ◽  
Feng Tian ◽  
...  
Keyword(s):  
Neural Network ◽  
Remote Sensing ◽  
State Of The Art ◽  
Complex Structure ◽  
Building Extraction ◽  
Training Process ◽  
Residual Network ◽  
Data Set ◽  
Deep Layers ◽  
The One

At present, convolutional neural networks (CNN) have been widely used in building extraction from remote sensing imagery (RSI), but there are still some bottlenecks. On the one hand, there are so many parameters in the previous network with complex structure, which will occupy lots of memories and consume much time during training process. On the other hand, low-level features extracted by shallow layers and abstract features extracted by deep layers of artificial neural network cannot be fully fused, which leads to an inaccurate building extraction from RSI. To alleviate these disadvantages, a dense residual neural network (DR-Net) was proposed in this paper. DR-Net uses a deeplabv3+Net encoder/decoder backbone, in combination with densely connected convolution neural network (DCNN) and residual network (ResNet) structure. Compared with deeplabv3+net (containing about 41 million parameters) and BRRNet (containing about 17 million parameters), DR-Net contains about 9 million parameters; So, the number of parameters reduced a lot. The experimental results for both the WHU Building Dataset and Massachusetts Building Dataset, DR-Net show better performance in building extraction than other two state-of-the-art methods. Experiments on WHU building data set showed that Intersection over Union (IoU) increased by 2.4% and F1 score increased by 1.4%; in terms of Massachusetts Building Dataset, IoU increased by 3.8% and F1 score increased by 2.9%.

scholarly journals Gated Autoencoder Network for Spectral–Spatial Hyperspectral Unmixing

2021 ◽  
Vol 13 (16) ◽  
pp. 3147
Author(s):  
Ziqiang Hua ◽  
Xiaorun Li ◽  
Jianfeng Jiang ◽  
Liaoying Zhao
Keyword(s):  
Real World ◽  
Spatial Information ◽  
State Of The Art ◽  
The State ◽  
Spectral Unmixing ◽  
Experimental Results ◽  
Spectral Information ◽  
Hyperspectral Unmixing ◽  
Spatial Features ◽  
Gating Mechanism

Convolution-based autoencoder networks have yielded promising performances in exploiting spatial–contextual signatures for spectral unmixing. However, the extracted spectral and spatial features of some networks are aggregated, which makes it difficult to balance their effects on unmixing results. In this paper, we propose two gated autoencoder networks with the intention of adaptively controlling the contribution of spectral and spatial features in unmixing process. Gating mechanism is adopted in the networks to filter and regularize spatial features to construct an unmixing algorithm based on spectral information and supplemented by spatial information. In addition, abundance sparsity regularization and gating regularization are introduced to ensure the appropriate implementation. Experimental results validate the superiority of the proposed method to the state-of-the-art techniques in both synthetic and real-world scenes. This study confirms the effectiveness of gating mechanism in improving the accuracy and efficiency of utilizing spatial signatures for spectral unmixing.

scholarly journals Deep residual network framework for structural health monitoring

2020 ◽  
pp. 147592172091837 ◽  
Author(s):  
Ruhua Wang ◽  
Chencho ◽  
Senjian An ◽  
Jun Li ◽  
Ling Li ◽  
...  
Keyword(s):  
Neural Network ◽  
Structural Health Monitoring ◽  
Convolutional Neural Network ◽  
Health Monitoring ◽  
Damage Identification ◽  
State Of The Art ◽  
Experimental Studies ◽  
The State ◽  
Residual Network ◽  
Structural Health

Convolutional neural networks have been widely employed for structural health monitoring and damage identification. The convolutional neural network is currently considered as the state-of-the-art method for structural damage identification due to its capabilities of efficient and robust feature learning in a hierarchical manner. It is a tendency to develop a convolutional neural network with a deeper architecture to gain a better performance. However, when the depth of the network increases to a certain level, the performance will degrade due to the gradient vanishing issue. Residual neural networks can avoid the problem of vanishing gradients by utilizing skip connections, which allows the information flowing to the next layer through identity mappings. In this article, a deep residual network framework is proposed for structural health monitoring of civil engineering structures. This framework is composed of purely residual blocks which operate as feature extractors and a fully connected layer as a regressor. It learns the damage-related features from the vibration characteristics such as mode shapes and maps them into the damage index labels, for example, stiffness reductions of structures. To evaluate the efficacy and robustness of the proposed framework, an intensive evaluation is conducted with both numerical and experimental studies. The comparison between the proposed approach and the state-of-the-art models, including a sparse autoencoder neural network, a shallow convolutional neural network and a convolutional neural network with the same structure but without skip connections, is conducted. In the numerical studies, a 7-storey steel frame is investigated. Four scenarios with considering measurement noise and finite element modelling errors in the data sets are studied. The proposed framework consistently outperforms the state-of-the-art models in all the scenarios, especially for the most challenging scenario, which includes both measurement noise and uncertainties. Experimental studies on a prestressed concrete bridge in the laboratory are conducted. The proposed framework demonstrates consistent damage prediction results on this beam with the state-of-the-art models.

scholarly journals PWNet: An Adaptive Weight Network for the Fusion of Panchromatic and Multispectral Images

2020 ◽  
Vol 12 (17) ◽  
pp. 2804
Author(s):  
Junmin Liu ◽  
Yunqiao Feng ◽  
Changsheng Zhou ◽  
Chunxia Zhang
Keyword(s):  
Neural Network ◽  
Spatial Resolution ◽  
High Spatial Resolution ◽  
State Of The Art ◽  
Spectral Information ◽  
Data Sets ◽  
Multispectral Images ◽  
Image Prior ◽  
Adaptive Weight ◽  
Component Substitution

Pansharpening is a typical image fusion problem, which aims to produce a high resolution multispectral (HRMS) image by integrating a high spatial resolution panchromatic (PAN) image with a low spatial resolution multispectral (MS) image. Prior arts have used either component substitution (CS)-based methods or multiresolution analysis (MRA)-based methods for this propose. Although they are simple and easy to implement, they usually suffer from spatial or spectral distortions and could not fully exploit the spatial and/or spectral information existed in PAN and MS images. By considering their complementary performances and with the goal of combining their advantages, we propose a pansharpening weight network (PWNet) to adaptively average the fusion results obtained by different methods. The proposed PWNet works by learning adaptive weight maps for different CS-based and MRA-based methods through an end-to-end trainable neural network (NN). As a result, the proposed PWN inherits the data adaptability or flexibility of NN, while maintaining the advantages of traditional methods. Extensive experiments on data sets acquired by three different kinds of satellites demonstrate the superiority of the proposed PWNet and its competitiveness with the state-of-the-art methods.

scholarly journals A New Architecture of Densely Connected Convolutional Networks for Pan-Sharpening

2020 ◽  
Vol 9 (4) ◽  
pp. 242
Author(s):  
Wei Huang ◽  
Jingjing Feng ◽  
Hua Wang ◽  
Le Sun
Keyword(s):  
Neural Network ◽  
Remote Sensing ◽  
Network Architecture ◽  
State Of The Art ◽  
Remote Sensing Image ◽  
Training Sample ◽  
Spectral Information ◽  
Convolutional Networks ◽  
Remote Sensing Image Fusion ◽  
Sample Set

In this paper, we propose a new architecture of densely connected convolutional networks for pan-sharpening (DCCNP). Since the traditional convolution neural network (CNN) has difficulty handling the lack of a training sample set in the field of remote sensing image fusion, it easily leads to overfitting and the vanishing gradient problem. Therefore, we employed an effective two-dense-block architecture to solve these problems. Meanwhile, to reduce the network architecture complexity, the batch normalization (BN) layer was removed in the design architecture of DenseNet. A new architecture of DenseNet for pan-sharpening, called DCCNP, is proposed, which uses a bottleneck layer and compression factors to narrow the network and reduce the network parameters, effectively suppressing overfitting. The experimental results show that the proposed method can yield a higher performance compared with other state-of-the-art pan-sharpening methods. The proposed method not only improves the spatial resolution of multi-spectral images, but also maintains the spectral information well.

scholarly journals Multilevel Attention Residual Neural Network for Multimodal Online Social Network Rumor Detection

2021 ◽  
Vol 9 ◽  
Author(s):  
Zhuang Wang ◽  
Jie Sui
Keyword(s):  
Neural Network ◽  
Social Network ◽  
State Of The Art ◽  
Online Social Network ◽  
Image Representation ◽  
Rapid Rise ◽  
Text And Image ◽  
Detection Model ◽  
Fused Image ◽  
Rumor Detection

In recent years, with the rapid rise of social networks, such as Weibo and Twitter, multimodal social network rumors have also spread. Unlike traditional unimodal rumor detection, the main difficulty of multimodal rumor detection is in avoiding the generation of noise information while using the complementarity of different modal features. In this article, we propose a multimodal online social network rumor detection model based on the multilevel attention residual neural network (MARN). First, the features of text and image are extracted by Bert and ResNet-18, respectively, and the cross-attention residual mechanism is used to enhance the representation of images with a text vector. Second, the enhanced image vector and text vector are concatenated and fused by the self-attention residual mechanism. Finally, the fused image–text vectors are classified into two categories. Among them, the attention mechanism can effectively enhance the image representation and further improve the fusion effect between the image and the text, while the residual mechanism retains the unique attributes of each original modal feature while using different modal features. To assess the performance of the MARN model, we conduct experiments on the Weibo dataset, and the results show that the MARN model outperforms the state-of-the-art models in terms of accuracy and F1 value.

scholarly journals Sentinel-2 Image Fusion Using a Deep Residual Network

2018 ◽  
Vol 10 (8) ◽  
pp. 1290 ◽  
Author(s):  
Frosti Palsson ◽  
Johannes Sveinsson ◽  
Magnus Ulfarsson
Keyword(s):  
Spatial Resolution ◽  
Near Infrared ◽  
State Of The Art ◽  
Residual Network ◽  
Fused Image ◽  
Coarse Spatial Resolution ◽  
Single Sensor ◽  
Shortwave Infrared ◽  
Sentinel 2

Single sensor fusion is the fusion of two or more spectrally disjoint reflectance bands that have different spatial resolution and have been acquired by the same sensor. An example is Sentinel-2, a constellation of two satellites, which can acquire multispectral bands of 10 m, 20 m and 60 m resolution for visible, near infrared (NIR) and shortwave infrared (SWIR). In this paper, we present a method to fuse the fine and coarse spatial resolution bands to obtain finer spatial resolution versions of the coarse bands. It is based on a deep convolutional neural network which has a residual design that models the fusion problem. The residual architecture helps the network to converge faster and allows for deeper networks by relieving the network of having to learn the coarse spatial resolution part of the inputs, enabling it to focus on constructing the missing fine spatial details. Using several real Sentinel-2 datasets, we study the effects of the most important hyperparameters on the quantitative quality of the fused image, compare the method to several state-of-the-art methods and demonstrate that it outperforms the comparison methods in experiments.

scholarly journals RecPOID: POI Recommendation with Friendship Aware and Deep CNN

2021 ◽  
Vol 13 (3) ◽  
pp. 79
Author(s):  
Sadaf Safavi ◽  
Mehrdad Jalali
Keyword(s):  
Neural Network ◽  
Convolutional Neural Network ◽  
State Of The Art ◽  
Spatial Features ◽  
Points Of Interest ◽  
Poi Recommendation ◽  
Time Distance ◽  
Deep Cnn ◽  
Temporal And Spatial ◽  
Location Based Social Networks

In location-based social networks (LBSNs), exploit several key features of points-of-interest (POIs) and users on precise POI recommendation be significant. In this work, a novel POI recommendation pipeline based on the convolutional neural network named RecPOID is proposed, which can recommend an accurate sequence of top-k POIs and considers only the effect of the most similar pattern friendship rather than all user’s friendship. We use the fuzzy c-mean clustering method to find the similarity. Temporal and spatial features of similar friends are fed to our Deep CNN model. The 10-layer convolutional neural network can predict longitude and latitude and the Id of the next proper locations; after that, based on the shortest time distance from a similar pattern’s friendship, select the smallest distance locations. The proposed structure uses six features, including user’s ID, month, day, hour, minute, and second of visiting time by each user as inputs. RecPOID based on two accessible LBSNs datasets is evaluated. Experimental outcomes illustrate considering most similar friendship could improve the accuracy of recommendations and the proposed RecPOID for POI recommendation outperforms state-of-the-art approaches.

Sign in / Sign up

Export Citation Format

Share Document