scholarly journals Double Ghost Convolution Attention Mechanism Network: A Framework for Hyperspectral Reconstruction of a Single RGB Image

Sensors ◽  
2021 ◽  
Vol 21 (2) ◽  
pp. 666
Author(s):  
Wenju Wang ◽  
Jiangwei Wang
Keyword(s):  
Deep Learning ◽  
Hyperspectral Image ◽  
Activation Function ◽  
Attention Mechanism ◽  
Spectral Accuracy ◽  
Reconstruction Accuracy ◽  
Spectral Reconstruction ◽  
Rgb Images ◽  
Feature Information ◽  
Rgb Image

Current research on the reconstruction of hyperspectral images from RGB images using deep learning mainly focuses on learning complex mappings through deeper and wider convolutional neural networks (CNNs). However, the reconstruction accuracy of the hyperspectral image is not high and among other issues the model for generating these images takes up too much storage space. In this study, we propose the double ghost convolution attention mechanism network (DGCAMN) framework for the reconstruction of a single RGB image to improve the accuracy of spectral reconstruction and reduce the storage occupied by the model. The proposed DGCAMN consists of a double ghost residual attention block (DGRAB) module and optimal nonlocal block (ONB). DGRAB module uses GhostNet and PRELU activation functions to reduce the calculation parameters of the data and reduce the storage size of the generative model. At the same time, the proposed double output feature Convolutional Block Attention Module (DOFCBAM) is used to capture the texture details on the feature map to maximize the content of the reconstructed hyperspectral image. In the proposed ONB, the Argmax activation function is used to obtain the region with the most abundant feature information and maximize the most useful feature parameters. This helps to improve the accuracy of spectral reconstruction. These contributions enable the DGCAMN framework to achieve the highest spectral accuracy with minimal storage consumption. The proposed method has been applied to the NTIRE 2020 dataset. Experimental results show that the proposed DGCAMN method outperforms the spectral accuracy reconstructed by advanced deep learning methods and greatly reduces storage consumption.

scholarly journals Physically Plausible Spectral Reconstruction

Sensors ◽  
2020 ◽  
Vol 20 (21) ◽  
pp. 6399
Author(s):  
Yi-Tun Lin ◽  
Graham D. Finlayson
Keyword(s):  
Deep Learning ◽  
State Of The Art ◽  
Reconstruction Algorithms ◽  
Spectral Accuracy ◽  
Recovery Method ◽  
Spectral Reconstruction ◽  
Training Stage ◽  
Spectral Recovery ◽  
Recovery Performance ◽  
Rgb Image

Spectral reconstruction algorithms recover spectra from RGB sensor responses. Recent methods—with the very best algorithms using deep learning—can already solve this problem with good spectral accuracy. However, the recovered spectra are physically incorrect in that they do not induce the RGBs from which they are recovered. Moreover, if the exposure of the RGB image changes then the recovery performance often degrades significantly—i.e., most contemporary methods only work for a fixed exposure. In this paper, we develop a physically accurate recovery method: the spectra we recover provably induce the same RGBs. Key to our approach is the idea that the set of spectra that integrate to the same RGB can be expressed as the sum of a unique fundamental metamer (spanned by the camera’s spectral sensitivities and linearly related to the RGB) and a linear combination of a vector space of metameric blacks (orthogonal to the spectral sensitivities). Physically plausible spectral recovery resorts to finding a spectrum that adheres to the fundamental metamer plus metameric black decomposition. To further ensure spectral recovery that is robust to changes in exposure, we incorporate exposure changes in the training stage of the developed method. In experiments we evaluate how well the methods recover spectra and predict the actual RGBs and RGBs under different viewing conditions (changing illuminations and/or cameras). The results show that our method generally improves the state-of-the-art spectral recovery (with more stabilized performance when exposure varies) and provides zero colorimetric error. Moreover, our method significantly improves the color fidelity under different viewing conditions, with up to a 60% reduction in some cases.

scholarly journals Classification of Hyperspectral Image Based on Double-Branch Dual-Attention Mechanism Network

2020 ◽  
Vol 12 (3) ◽  
pp. 582 ◽  
Author(s):  
Rui Li ◽  
Shunyi Zheng ◽  
Chenxi Duan ◽  
Yang Yang ◽  
Xiqi Wang
Keyword(s):  
Deep Learning ◽  
Hyperspectral Image ◽  
State Of The Art ◽  
Attention Mechanism ◽  
Superior Performance ◽  
Feature Maps ◽  
Spatial Features ◽  
Training Samples ◽  
Series Of Experiments

In recent years, researchers have paid increasing attention on hyperspectral image (HSI) classification using deep learning methods. To improve the accuracy and reduce the training samples, we propose a double-branch dual-attention mechanism network (DBDA) for HSI classification in this paper. Two branches are designed in DBDA to capture plenty of spectral and spatial features contained in HSI. Furthermore, a channel attention block and a spatial attention block are applied to these two branches respectively, which enables DBDA to refine and optimize the extracted feature maps. A series of experiments on four hyperspectral datasets show that the proposed framework has superior performance to the state-of-the-art algorithm, especially when the training samples are signally lacking.

scholarly journals Multiscale Deep Spatial Feature Extraction Using Virtual RGB Image for Hyperspectral Imagery Classification

2020 ◽  
Vol 12 (2) ◽  
pp. 280 ◽  
Author(s):  
Liqin Liu ◽  
Zhenwei Shi ◽  
Bin Pan ◽  
Ning Zhang ◽  
Huanlin Luo ◽  
...  
Keyword(s):  
Feature Extraction ◽  
Deep Learning ◽  
Hyperspectral Image ◽  
Feature Fusion ◽  
Hyperspectral Data ◽  
Learning Technology ◽  
Learning Networks ◽  
Spatial Features ◽  
Training Samples ◽  
Rgb Image

In recent years, deep learning technology has been widely used in the field of hyperspectral image classification and achieved good performance. However, deep learning networks need a large amount of training samples, which conflicts with the limited labeled samples of hyperspectral images. Traditional deep networks usually construct each pixel as a subject, ignoring the integrity of the hyperspectral data and the methods based on feature extraction are likely to lose the edge information which plays a crucial role in the pixel-level classification. To overcome the limit of annotation samples, we propose a new three-channel image build method (virtual RGB image) by which the trained networks on natural images are used to extract the spatial features. Through the trained network, the hyperspectral data are disposed as a whole. Meanwhile, we propose a multiscale feature fusion method to combine both the detailed and semantic characteristics, thus promoting the accuracy of classification. Experiments show that the proposed method can achieve ideal results better than the state-of-art methods. In addition, the virtual RGB image can be extended to other hyperspectral processing methods that need to use three-channel images.

MCA-Unet: Multi-class Attention-aware U-net for Seismic Phase Picking

2021 ◽  
Author(s):  
Wei Li ◽  
Georg Rümpker ◽  
Horst Stöcker ◽  
Megha Chakraborty ◽  
Darius Fener ◽  
...  
Keyword(s):  
Deep Learning ◽  
Network Architecture ◽  
Attention Mechanism ◽  
Seismic Waveforms ◽  
Decoder Architecture ◽  
Seismic Event Detection ◽  
Feature Information ◽  
Multi Scales ◽  
Segmentation Problem ◽  
Different Levels

<p>This study presents a deep learning based algorithm for seismic event detection and simultaneous phase picking in seismic waveforms. U-net structure-based solutions which consists of a contracting path (encoder) to capture feature information and a symmetric expanding path (decoder) that enables precise localization, have proven to be effective in phase picking. The network architecture of these U-net models mainly comprise of 1D CNN, Bi- & Uni-directional LSTM, transformers and self-attentive layers. Althought, these networks have proven to be a good solution, they may not fully harness the information extracted from multi-scales.</p><p> In this study, we propose a simple yet powerful deep learning architecture by combining multi-class with attention mechanism, named MCA-Unet, for phase picking.  Specially, we treat the phase picking as an image segmentation problem, and incorporate the attention mechanism into the U-net structure to efficiently deal with the features extracted at different levels with the goal to improve the performance on the seismic phase picking. Our neural network is based on an encoder-decoder architecture composed of 1D convolutions, pooling layers, deconvolutions and multi-attention layers. This architecture is applied and tested to a field seismic dataset (e.g. Wenchuan Earthquake Aftershocks Classification Dataset) to check its performance.</p>

scholarly journals A Lightweight 1-D Convolution Augmented Transformer with Metric Learning for Hyperspectral Image Classification

Sensors ◽  
2021 ◽  
Vol 21 (5) ◽  
pp. 1751
Author(s):  
Xiang Hu ◽  
Wenjing Yang ◽  
Hao Wen ◽  
Yu Liu ◽  
Yuanxi Peng
Keyword(s):  
Computer Vision ◽  
Deep Learning ◽  
Image Classification ◽  
Language Processing ◽  
Hyperspectral Image ◽  
Metric Learning ◽  
Image Data ◽  
Activation Function ◽  
Classification Performance ◽  
Hyperspectral Image Classification

Hyperspectral image (HSI) classification is the subject of intense research in remote sensing. The tremendous success of deep learning in computer vision has recently sparked the interest in applying deep learning in hyperspectral image classification. However, most deep learning methods for hyperspectral image classification are based on convolutional neural networks (CNN). Those methods require heavy GPU memory resources and run time. Recently, another deep learning model, the transformer, has been applied for image recognition, and the study result demonstrates the great potential of the transformer network for computer vision tasks. In this paper, we propose a model for hyperspectral image classification based on the transformer, which is widely used in natural language processing. Besides, we believe we are the first to combine the metric learning and the transformer model in hyperspectral image classification. Moreover, to improve the model classification performance when the available training samples are limited, we use the 1-D convolution and Mish activation function. The experimental results on three widely used hyperspectral image data sets demonstrate the proposed model’s advantages in accuracy, GPU memory cost, and running time.

scholarly journals Sentinel2GlobalLULC: A deep-learning-ready Sentinel-2 RGB image dataset for global land use/cover mapping

2021 ◽  
Author(s):  
Yassir Benhammou ◽  
Domingo Alcaraz-Segura ◽  
Emilio Guirado ◽  
Rohaifa Khaldi ◽  
Boujemâa Achchab ◽  
...  
Keyword(s):  
Land Use ◽  
Deep Learning ◽  
Remote Sensing Data ◽  
Google Earth ◽  
Climate Modelling ◽  
Human Modification ◽  
Image Dataset ◽  
Rgb Images ◽  
Rgb Image ◽  
Sentinel 2

ABSTRACTLand-Use and Land-Cover (LULC) mapping is relevant for many applications, from Earth system and climate modelling to territorial and urban planning. Global LULC products are continuously developing as remote sensing data and methods grow. However, there is still low consistency among LULC products due to low accuracy for some regions and LULC types. Here, we introduce Sentinel2GlobalLULC, a Sentinel-2 RGB image dataset, built from the consensus of 15 global LULC maps available in Google Earth Engine. Sentinel2GlobalLULC v1.1 contains 195572 RGB images organized into 29 global LULC mapping classes. Each image is a tile that has 224 × 224 pixels at 10 × 10 m spatial resolution and was built as a cloud-free composite from all Sentinel-2 images acquired between June 2015 and October 2020. Metadata includes a unique LULC type annotation per image, together with level of consensus, reverse geo-referencing, and global human modification index. Sentinel2GlobalLULC is optimized for the state-of-the-art Deep Learning models to provide a new gate towards building precise and robust global or regional LULC maps.

scholarly journals Deep Unsupervised Fusion Learning for Hyperspectral Image Super Resolution

Sensors ◽  
2021 ◽  
Vol 21 (7) ◽  
pp. 2348
Author(s):  
Zhe Liu ◽  
Yinqiang Zheng ◽  
Xian-Hua Han
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Network Architecture ◽  
Hyperspectral Image ◽  
Super Resolution ◽  
Learning Framework ◽  
Reconstruction Performance ◽  
Rgb Images ◽  
Image Super Resolution ◽  
Image Priors

Hyperspectral image (HSI) super-resolution (SR) is a challenging task due to its ill-posed nature, and has attracted extensive attention by the research community. Previous methods concentrated on leveraging various hand-crafted image priors of a latent high-resolution hyperspectral (HR-HS) image to regularize the degradation model of the observed low-resolution hyperspectral (LR-HS) and HR-RGB images. Different optimization strategies for searching a plausible solution, which usually leads to a limited reconstruction performance, were also exploited. Recently, deep-learning-based methods evolved for automatically learning the abundant image priors in a latent HR-HS image. These methods have made great progress for HS image super resolution. Current deep-learning methods have faced difficulties in designing more complicated and deeper neural network architectures for boosting the performance. They also require large-scale training triplets, such as the LR-HS, HR-RGB, and their corresponding HR-HS images for neural network training. These training triplets significantly limit their applicability to real scenarios. In this work, a deep unsupervised fusion-learning framework for generating a latent HR-HS image using only the observed LR-HS and HR-RGB images without previous preparation of any other training triplets is proposed. Based on the fact that a convolutional neural network architecture is capable of capturing a large number of low-level statistics (priors) of images, the automatic learning of underlying priors of spatial structures and spectral attributes in a latent HR-HS image using only its corresponding degraded observations is promoted. Specifically, the parameter space of a generative neural network used for learning the required HR-HS image to minimize the reconstruction errors of the observations using mathematical relations between data is investigated. Moreover, special convolutional layers for approximating the degradation operations between observations and the latent HR-HS image are specifically to construct an end-to-end unsupervised learning framework for HS image super-resolution. Experiments on two benchmark HS datasets, including the CAVE and Harvard, demonstrate that the proposed method can is capable of producing very promising results, even under a large upscaling factor. Furthermore, it can outperform other unsupervised state-of-the-art methods by a large margin, and manifests its superiority and efficiency.

scholarly journals Generative Adversarial Networks Based on Collaborative Learning and Attention Mechanism for Hyperspectral Image Classification

2020 ◽  
Vol 12 (7) ◽  
pp. 1149 ◽  
Author(s):  
Jie Feng ◽  
Xueliang Feng ◽  
Jiantong Chen ◽  
Xianghai Cao ◽  
Xiangrong Zhang ◽  
...  
Keyword(s):  
Collaborative Learning ◽  
Hyperspectral Image ◽  
Small Sample Size ◽  
Activation Function ◽  
Classification Performance ◽  
Small Sample ◽  
Attention Mechanism ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Convolutional Network

Classifying hyperspectral images (HSIs) with limited samples is a challenging issue. The generative adversarial network (GAN) is a promising technique to mitigate the small sample size problem. GAN can generate samples by the competition between a generator and a discriminator. However, it is difficult to generate high-quality samples for HSIs with complex spatial–spectral distribution, which may further degrade the performance of the discriminator. To address this problem, a symmetric convolutional GAN based on collaborative learning and attention mechanism (CA-GAN) is proposed. In CA-GAN, the generator and the discriminator not only compete but also collaborate. The shallow to deep features of real multiclass samples in the discriminator assist the sample generation in the generator. In the generator, a joint spatial–spectral hard attention module is devised by defining a dynamic activation function based on a multi-branch convolutional network. It impels the distribution of generated samples to approximate the distribution of real HSIs both in spectral and spatial dimensions, and it discards misleading and confounding information. In the discriminator, a convolutional LSTM layer is merged to extract spatial contextual features and capture long-term spectral dependencies simultaneously. Finally, the classification performance of the discriminator is improved by enforcing competitive and collaborative learning between the discriminator and generator. Experiments on HSI datasets show that CA-GAN obtains satisfactory classification results compared with advanced methods, especially when the number of training samples is limited.

scholarly journals Classification of Hyperspectral Image Based on Double-Branch Dual-Attention Mechanism Network

2020 ◽  
Author(s):  
Rui Li ◽  
Shunyi Zheng ◽  
Chenxi Duan ◽  
Yang Yang ◽  
Xiqi Wang
Keyword(s):  
Deep Learning ◽  
Hyperspectral Image ◽  
State Of The Art ◽  
Attention Mechanism ◽  
Superior Performance ◽  
Feature Maps ◽  
Spatial Features ◽  
Training Samples ◽  
Series Of Experiments

In recent years, researchers have paid increasing attention on hyperspectral image (HSI) classification using deep learning methods. To improve the accuracy and reduce the training samples, we propose a double-branch dual-attention mechanism network (DBDA) for HSI classification in this paper. Two branches are designed in DBDA to capture plenty of spectral and spatial features contained in HSI. Furthermore, a channel attention block and a spatial attention block are applied to these two branches respectively, which enables DBDA to refine and optimize the extracted feature maps. A series of experiments on four hyperspectral datasets show that the proposed framework has superior performance to the state-of-the-art algorithm, especially when the training samples are signally lacking.

A Review on the Attention Mechanism of Deep Learning

2021 ◽  
Author(s):  
Zhaoyang Niu ◽  
Guoqiang Zhong ◽  
Hui Yu
Keyword(s):  
Deep Learning ◽  
Attention Mechanism
Sign in / Sign up

Export Citation Format

Share Document