scholarly journals Hyperspectral Pansharpening Based on Improved Deep Image Prior and Residual Reconstruction

2021 ◽  
Author(s):  
Wele Gedara Chaminda Bandara ◽  
Jeya Maria Jose Valanarasu ◽  
Vishal M. Patel
Keyword(s):  
Hyperspectral Image ◽  
Spectral Response ◽  
Large Data ◽  
Spatial Domain ◽  
Convolutional Networks ◽  
Image Prior ◽  
Residual Image ◽  
Deep Layers ◽  
Deep Image ◽  
High Level

<div> \par Hyperspectral pansharpening aims to synthesize a low-resolution hyperspectral image (LR-HSI) with a registered panchromatic image (PAN) to generate an enhanced HSI with high spectral and spatial resolution. Recently proposed HS pansharpening methods have obtained remarkable results using deep convolutional networks (ConvNets), which typically consist of three steps: (1) up-sampling the LR-HSI, (2) predicting the residual image via a ConvNet, and (3) obtaining the final fused HSI by adding the outputs from first and second steps. Recent methods have leveraged Deep Image Prior (DIP) to up-sample the LR-HSI due to its excellent ability to preserve both spatial and spectral information, without learning from large data sets. However, we observed that the quality of up-sampled HSIs can be further improved by introducing an additional spatial-domain constraint to the conventional spectral-domain energy function. We define our spatial-domain constraint as the $L_1$ distance between the predicted PAN image and the actual PAN image. To estimate the PAN image of the up-sampled HSI, we also propose a learnable spectral response function (SRF). Moreover, we noticed that the residual image between the up-sampled HSI and the reference HSI mainly consists of edge information and very fine structures. In order to accurately estimate fine information, we propose a novel over-complete network, called HyperKite, which focuses on learning high-level features by constraining the receptive from increasing in the deep layers. We perform experiments on three HSI datasets to demonstrate the superiority of our DIP-HyperKite over the state-of-the-art pansharpening methods. The deployment codes, pre-trained models, and final fusion outputs of our DIP-HyperKite and the methods used for the comparisons will be publicly made available at \url{https://github.com/wgcban/DIP-HyperKite.git}</div><div><br></div>

scholarly journals Hyperspectral Pansharpening Based on Improved Deep Image Prior and Residual Reconstruction

2021 ◽  
Author(s):  
Wele Gedara Chaminda Bandara ◽  
Jeya Maria Jose Valanarasu ◽  
Vishal M. Patel
Keyword(s):  
Hyperspectral Image ◽  
Spectral Response ◽  
Large Data ◽  
Spatial Domain ◽  
Convolutional Networks ◽  
Image Prior ◽  
Residual Image ◽  
Deep Layers ◽  
Deep Image ◽  
High Level

<div> \par Hyperspectral pansharpening aims to synthesize a low-resolution hyperspectral image (LR-HSI) with a registered panchromatic image (PAN) to generate an enhanced HSI with high spectral and spatial resolution. Recently proposed HS pansharpening methods have obtained remarkable results using deep convolutional networks (ConvNets), which typically consist of three steps: (1) up-sampling the LR-HSI, (2) predicting the residual image via a ConvNet, and (3) obtaining the final fused HSI by adding the outputs from first and second steps. Recent methods have leveraged Deep Image Prior (DIP) to up-sample the LR-HSI due to its excellent ability to preserve both spatial and spectral information, without learning from large data sets. However, we observed that the quality of up-sampled HSIs can be further improved by introducing an additional spatial-domain constraint to the conventional spectral-domain energy function. We define our spatial-domain constraint as the $L_1$ distance between the predicted PAN image and the actual PAN image. To estimate the PAN image of the up-sampled HSI, we also propose a learnable spectral response function (SRF). Moreover, we noticed that the residual image between the up-sampled HSI and the reference HSI mainly consists of edge information and very fine structures. In order to accurately estimate fine information, we propose a novel over-complete network, called HyperKite, which focuses on learning high-level features by constraining the receptive from increasing in the deep layers. We perform experiments on three HSI datasets to demonstrate the superiority of our DIP-HyperKite over the state-of-the-art pansharpening methods. The deployment codes, pre-trained models, and final fusion outputs of our DIP-HyperKite and the methods used for the comparisons will be publicly made available at \url{https://github.com/wgcban/DIP-HyperKite.git}</div><div><br></div>

scholarly journals Real-Time Adversarial Attack Detection with Deep Image Prior Initialized as a High-Level Representation Based Blurring Network

Electronics ◽  
2020 ◽  
Vol 10 (1) ◽  
pp. 52
Author(s):  
Richard Evan Sutanto ◽  
Sukho Lee
Keyword(s):  
Neural Network ◽  
Attack Detection ◽  
Detection Methods ◽  
Defense System ◽  
Image Prior ◽  
The Neural Network ◽  
Adversarial Examples ◽  
Deep Image ◽  
Adversarial Attack ◽  
High Level

Several recent studies have shown that artificial intelligence (AI) systems can malfunction due to intentionally manipulated data coming through normal channels. Such kinds of manipulated data are called adversarial examples. Adversarial examples can pose a major threat to an AI-led society when an attacker uses them as means to attack an AI system, which is called an adversarial attack. Therefore, major IT companies such as Google are now studying ways to build AI systems which are robust against adversarial attacks by developing effective defense methods. However, one of the reasons why it is difficult to establish an effective defense system is due to the fact that it is difficult to know in advance what kind of adversarial attack method the opponent is using. Therefore, in this paper, we propose a method to detect the adversarial noise without knowledge of the kind of adversarial noise used by the attacker. For this end, we propose a blurring network that is trained only with normal images and also use it as an initial condition of the Deep Image Prior (DIP) network. This is in contrast to other neural network based detection methods, which require the use of many adversarial noisy images for the training of the neural network. Experimental results indicate the validity of the proposed method.

A deep image prior-based interpretable network for hyperspectral image fusion

2021 ◽  
Vol 12 (12) ◽  
pp. 1250-1259
Author(s):  
Yanglin Sun ◽  
Jianjun Liu ◽  
Jinlong Yang ◽  
Zhiyong Xiao ◽  
Zebin Wu
Keyword(s):  
Image Fusion ◽  
Hyperspectral Image ◽  
Image Prior ◽  
Deep Image ◽  
Hyperspectral Image Fusion

scholarly journals Efficient End-to-End Sentence-Level Lipreading with Temporal Convolutional Networks

2021 ◽  
Vol 11 (15) ◽  
pp. 6975
Author(s):  
Tao Zhang ◽  
Lun He ◽  
Xudong Li ◽  
Guoqing Feng
Keyword(s):  
Performance Improvement ◽  
State Of The Art ◽  
Error Rates ◽  
Convolutional Network ◽  
Convolutional Networks ◽  
Sentence Level ◽  
End To End ◽  
High Level ◽  
Improved Accuracy ◽  
Talking Face

Lipreading aims to recognize sentences being spoken by a talking face. In recent years, the lipreading method has achieved a high level of accuracy on large datasets and made breakthrough progress. However, lipreading is still far from being solved, and existing methods tend to have high error rates on the wild data and have the defects of disappearing training gradient and slow convergence. To overcome these problems, we proposed an efficient end-to-end sentence-level lipreading model, using an encoder based on a 3D convolutional network, ResNet50, Temporal Convolutional Network (TCN), and a CTC objective function as the decoder. More importantly, the proposed architecture incorporates TCN as a feature learner to decode feature. It can partly eliminate the defects of RNN (LSTM, GRU) gradient disappearance and insufficient performance, and this yields notable performance improvement as well as faster convergence. Experiments show that the training and convergence speed are 50% faster than the state-of-the-art method, and improved accuracy by 2.4% on the GRID dataset.

scholarly journals Hyperspectral Data Simulation (Sentinel-2 to AVIRIS-NG) for Improved Wildfire Fuel Mapping, Boreal Alaska

2021 ◽  
Vol 13 (9) ◽  
pp. 1693
Author(s):  
Anushree Badola ◽  
Santosh K. Panda ◽  
Dar A. Roberts ◽  
Christine F. Waigl ◽  
Uma S. Bhatt ◽  
...  
Keyword(s):  
Land Cover ◽  
Spectral Resolution ◽  
Hyperspectral Image ◽  
Spectral Response ◽  
Spectral Characteristics ◽  
Hyperspectral Data ◽  
Spectral Unmixing ◽  
High Spectral Resolution ◽  
Fuel Mapping ◽  
Sentinel 2

Alaska has witnessed a significant increase in wildfire events in recent decades that have been linked to drier and warmer summers. Forest fuel maps play a vital role in wildfire management and risk assessment. Freely available multispectral datasets are widely used for land use and land cover mapping, but they have limited utility for fuel mapping due to their coarse spectral resolution. Hyperspectral datasets have a high spectral resolution, ideal for detailed fuel mapping, but they are limited and expensive to acquire. This study simulates hyperspectral data from Sentinel-2 multispectral data using the spectral response function of the Airborne Visible/Infrared Imaging Spectrometer-Next Generation (AVIRIS-NG) sensor, and normalized ground spectra of gravel, birch, and spruce. We used the Uniform Pattern Decomposition Method (UPDM) for spectral unmixing, which is a sensor-independent method, where each pixel is expressed as the linear sum of standard reference spectra. The simulated hyperspectral data have spectral characteristics of AVIRIS-NG and the reflectance properties of Sentinel-2 data. We validated the simulated spectra by visually and statistically comparing it with real AVIRIS-NG data. We observed a high correlation between the spectra of tree classes collected from AVIRIS-NG and simulated hyperspectral data. Upon performing species level classification, we achieved a classification accuracy of 89% for the simulated hyperspectral data, which is better than the accuracy of Sentinel-2 data (77.8%). We generated a fuel map from the simulated hyperspectral image using the Random Forest classifier. Our study demonstrated that low-cost and high-quality hyperspectral data can be generated from Sentinel-2 data using UPDM for improved land cover and vegetation mapping in the boreal forest.

scholarly journals Efficient Lossless Compression of Multitemporal Hyperspectral Image Data

2018 ◽  
Vol 4 (12) ◽  
pp. 142 ◽  
Author(s):  
Hongda Shen ◽  
Zhuocheng Jiang ◽  
W. Pan
Keyword(s):  
Data Compression ◽  
Hyperspectral Image ◽  
Image Data ◽  
Lossless Compression ◽  
Large Data ◽  
Hyperspectral Data ◽  
Compression Performance ◽  
Temporal Correlations ◽  
Sensing Applications ◽  
Data Volume

Hyperspectral imaging (HSI) technology has been used for various remote sensing applications due to its excellent capability of monitoring regions-of-interest over a period of time. However, the large data volume of four-dimensional multitemporal hyperspectral imagery demands massive data compression techniques. While conventional 3D hyperspectral data compression methods exploit only spatial and spectral correlations, we propose a simple yet effective predictive lossless compression algorithm that can achieve significant gains on compression efficiency, by also taking into account temporal correlations inherent in the multitemporal data. We present an information theoretic analysis to estimate potential compression performance gain with varying configurations of context vectors. Extensive simulation results demonstrate the effectiveness of the proposed algorithm. We also provide in-depth discussions on how to construct the context vectors in the prediction model for both multitemporal HSI and conventional 3D HSI data.

Delving into Deep Image Prior for Adversarial Defense

2021 ◽  
Author(s):  
Li Ding ◽  
Yongwei Wang ◽  
Xin Ding ◽  
Kaiwen Yuan ◽  
Ping Wang ◽  
...  
Keyword(s):  
Image Prior ◽  
Deep Image

scholarly journals Automated monitoring system for events detection in sewer network by distribution temperature sensing data measurement

2018 ◽  
Vol 78 (7) ◽  
pp. 1499-1508
Author(s):  
Abdelhak Kessili ◽  
Jes Vollertsen ◽  
Asbjørn Haaning Nielsen
Keyword(s):  
Large Data ◽  
Temperature Sensing ◽  
Wastewater Discharge ◽  
Sewer Pipe ◽  
Temperature Changes ◽  
Set Up ◽  
Comparison Of The Results ◽  
Events Detection ◽  
High Level ◽  
Automated Tool

Abstract This study is related to distribution temperature sensing (DTS) in sewers for tracing illicit or unintended inflows to foul sewers. A DTS measurement is performed with a fiber optic cable that is installed at the invert of a sewer pipe in combination with a standalone laser/computer instrument. This set-up generates in-sewer temperature measurements with high resolutions in time (every minute) and space (every metre) along the cable over long periods of time (weeks on end). The prolonged monitoring period in combination with the high level of detail in the dataset allows the study of anomalies (i.e., unexpected temperatures and/or temperature variations at certain locations), even if these only occur very infrequently. The objective of this paper is to develop an automated tool to analyze the large data masses and identify anomalies caused by illicit or unintended inflows. In this study, an algorithm for detecting the temperature changes that are caused by both wastewater discharge and inflow of stormwater are developed. A comparison of the results of the automated procedure to the results of a manual assessment of the datasets (Elmehaven, Denmark) shows that the automated procedure performs very well.

Sign in / Sign up

Export Citation Format

Share Document