scholarly journals Multi-Temporal Sentinel-1 and -2 Data Fusion for Optical Image Simulation

2018 ◽  
Vol 7 (10) ◽  
pp. 389 ◽  
Author(s):  
Wei He ◽  
Naoto Yokoya
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Super Resolution ◽  
Optical Image ◽  
Optical Data ◽  
Image Simulation ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Multi Temporal ◽  
Sar Data

In this paper, we present the optical image simulation from synthetic aperture radar (SAR) data using deep learning based methods. Two models, i.e., optical image simulation directly from the SAR data and from multi-temporal SAR-optical data, are proposed to testify the possibilities. The deep learning based methods that we chose to achieve the models are a convolutional neural network (CNN) with a residual architecture and a conditional generative adversarial network (cGAN). We validate our models using the Sentinel-1 and -2 datasets. The experiments demonstrate that the model with multi-temporal SAR-optical data can successfully simulate the optical image; meanwhile, the state-of-the-art model with simple SAR data as input failed. The optical image simulation results indicate the possibility of SAR-optical information blending for the subsequent applications such as large-scale cloud removal, and optical data temporal super-resolution. We also investigate the sensitivity of the proposed models against the training samples, and reveal possible future directions.

scholarly journals MSG-GAN-SD: A Multi-Scale Gradients GAN for Statistical Downscaling of 2-Meter Temperature over the EURO-CORDEX Domain

AI ◽  
2021 ◽  
Vol 2 (4) ◽  
pp. 600-620
Author(s):  
Gabriele Accarino ◽  
Marco Chiarelli ◽  
Francesco Immorlano ◽  
Valeria Aloisi ◽  
Andrea Gatto ◽  
...  
Keyword(s):  
Deep Learning ◽  
Statistical Downscaling ◽  
Large Scale ◽  
Super Resolution ◽  
Climate Science ◽  
Cost Effective ◽  
Temperature Fields ◽  
Generative Adversarial Network ◽  
Multi Scale ◽  
Adversarial Network

One of the most important open challenges in climate science is downscaling. It is a procedure that allows making predictions at local scales, starting from climatic field information available at large scale. Recent advances in deep learning provide new insights and modeling solutions to tackle downscaling-related tasks by automatically learning the coarse-to-fine grained resolution mapping. In particular, deep learning models designed for super-resolution problems in computer vision can be exploited because of the similarity between images and climatic fields maps. For this reason, a new architecture tailored for statistical downscaling (SD), named MSG-GAN-SD, has been developed, allowing interpretability and good stability during training, due to multi-scale gradient information. The proposed architecture, based on a Generative Adversarial Network (GAN), was applied to downscale ERA-Interim 2-m temperature fields, from 83.25 to 13.87 km resolution, covering the EURO-CORDEX domain within the 1979–2018 period. The training process involves seasonal and monthly dataset arrangements, in addition to different training strategies, leading to several models. Furthermore, a model selection framework is introduced in order to mathematically select the best models during the training. The selected models were then tested on the 2015–2018 period using several metrics to identify the best training strategy and dataset arrangement, which finally produced several evaluation maps. This work is the first attempt to use the MSG-GAN architecture for statistical downscaling. The achieved results demonstrate that the models trained on seasonal datasets performed better than those trained on monthly datasets. This study presents an accurate and cost-effective solution that is able to perform downscaling of 2 m temperature climatic maps.

scholarly journals AWGAN: A Powerful Batch Correction Model for scRNA-seq Data

2021 ◽  
Author(s):  
Tianyu Liu ◽  
Yuge Wang ◽  
Hong-yu Zhao
Keyword(s):  
Deep Learning ◽  
Large Scale ◽  
Batch Effects ◽  
Generative Adversarial Network ◽  
Correction Model ◽  
Batch Correction ◽  
Learning Framework ◽  
Adversarial Network ◽  
Genomics Research ◽  
Downstream Analysis

With the advancement of technology, we can generate and access large-scale, high dimensional and diverse genomics data, especially through single-cell RNA sequencing (scRNA-seq). However, integrative downstream analysis from multiple scRNA-seq datasets remains challenging due to batch effects. In this paper, we focus on scRNA-seq data integration and propose a new deep learning framework based on Wasserstein Generative Adversarial Network (WGAN) combined with an attention mechanism to reduce the differences among batches. We also discuss the limitations of the existing methods and demonstrate the advantages of our new model from both theoretical and practical aspects, advocating the use of deep learning in genomics research.

scholarly journals Extreme Low-Resolution Activity Recognition Using a Super-Resolution-Oriented Generative Adversarial Network

Micromachines ◽  
2021 ◽  
Vol 12 (6) ◽  
pp. 670
Author(s):  
Mingzheng Hou ◽  
Song Liu ◽  
Jiliu Zhou ◽  
Yi Zhang ◽  
Ziliang Feng
Keyword(s):  
Activity Recognition ◽  
Large Scale ◽  
Super Resolution ◽  
General Activity ◽  
Low Resolution ◽  
Generative Adversarial Network ◽  
Video Clips ◽  
Adversarial Network ◽  
Efficient Recognition ◽  
Low Resolution Images

Activity recognition is a fundamental and crucial task in computer vision. Impressive results have been achieved for activity recognition in high-resolution videos, but for extreme low-resolution videos, which capture the action information at a distance and are vital for preserving privacy, the performance of activity recognition algorithms is far from satisfactory. The reason is that extreme low-resolution (e.g., 12 × 16 pixels) images lack adequate scene and appearance information, which is needed for efficient recognition. To address this problem, we propose a super-resolution-driven generative adversarial network for activity recognition. To fully take advantage of the latent information in low-resolution images, a powerful network module is employed to super-resolve the extremely low-resolution images with a large scale factor. Then, a general activity recognition network is applied to analyze the super-resolved video clips. Extensive experiments on two public benchmarks were conducted to evaluate the effectiveness of our proposed method. The results demonstrate that our method outperforms several state-of-the-art low-resolution activity recognition approaches.

scholarly journals CBNWI-50: A Deep Learning Bird Dataset for Image Translation and Resolution Improvement using Generative Adversarial Network

2019 ◽  
Vol 8 (9S) ◽  
pp. 91-102
Keyword(s):  
Deep Learning ◽  
Super Resolution ◽  
Generative Adversarial Networks ◽  
Western India ◽  
Generative Adversarial Network ◽  
Style Transfer ◽  
Adversarial Network ◽  
Image Translation ◽  
Common Birds ◽  
Single Image Super Resolution

Generative Adversarial Networks have gained prominence in a short span of time as they can synthesize images from latent noise by minimizing the adversarial cost function. New variants of GANs have been developed to perform specific tasks using state-of-the-art GAN models, like image translation, single image super resolution, segmentation, classification, style transfer etc. However, a combination of two GANs to perform two different applications in one model has been sparsely explored. Hence, this paper concatenates two GANs and aims to perform Image Translation using Cycle GAN model on bird images and improve their resolution using SRGAN. During the extensive survey, it is observed that most of the deep learning databases on Aves were built using the new world species (i.e. species found in North America). Hence, to bridge this gap, a new Ave database, 'Common Birds of North - Western India' (CBNWI-50), is also proposed in this work.

scholarly journals Edge Loss for Remote Sensing Image Super-Resolution

2021 ◽  
Author(s):  
Jiaoyue Li ◽  
Weifeng Liu ◽  
Kai Zhang ◽  
Baodi Liu
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Super Resolution ◽  
Ground Truth ◽  
Remote Sensing Image ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Sensing Applications ◽  
Image Super Resolution ◽  
Edge Based

Remote sensing image super-resolution (SR) plays an essential role in many remote sensing applications. Recently, remote sensing image super-resolution methods based on deep learning have shown remarkable performance. However, directly utilizing the deep learning methods becomes helpless to recover the remote sensing images with a large number of complex objectives or scene. So we propose an edge-based dense connection generative adversarial network (SREDGAN), which minimizes the edge differences between the generated image and its corresponding ground truth. Experimental results on NWPU-VHR-10 and UCAS-AOD datasets demonstrate that our method improves 1.92 and 0.045 in PSNR and SSIM compared with SRGAN, respectively.

scholarly journals SRGAN Assisted Encoder-Decoder Deep Neural Network for Colorectal Polyp Semantic Segmentation

2021 ◽  
Vol 35 (5) ◽  
pp. 395-401
Author(s):  
Mohan Mahanty ◽  
Debnath Bhattacharyya ◽  
Divya Midhunchakkaravarthy
Keyword(s):  
Deep Learning ◽  
Standard Procedure ◽  
Super Resolution ◽  
Semantic Segmentation ◽  
Colorectal Polyp ◽  
Polyp Detection ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Colon Cancers ◽  
Benchmark Datasets

Colon cancer is thought about as the third most regularly identified cancer after Brest and lung cancer. Most colon cancers are adenocarcinomas developing from adenomatous polyps, grow on the intima of the colon. The standard procedure for polyp detection is colonoscopy, where the success of the standard colonoscopy depends on the colonoscopist experience and other environmental factors. Nonetheless, throughout colonoscopy procedures, a considerable number (8-37%) of polyps are missed due to human mistakes, and these missed polyps are the prospective reason for colorectal cancer cells. In the last few years, many research groups developed deep learning-based computer-aided (CAD) systems that recommended many techniques for automated polyp detection, localization, and segmentation. Still, accurate polyp detection, segmentation is required to minimize polyp miss out rates. This paper suggested a Super-Resolution Generative Adversarial Network (SRGAN) assisted Encoder-Decoder network for fully automated colon polyp segmentation from colonoscopic images. The proposed deep learning model incorporates the SRGAN in the up-sampling process to achieve more accurate polyp segmentation. We examined our model on the publicly available benchmark datasets CVC-ColonDB and Warwick- QU. The model accomplished a dice score of 0.948 on the CVC-ColonDB dataset, surpassed the recently advanced state-of-the-art (SOTA) techniques. When it is evaluated on the Warwick-QU dataset, it attains a Dice Score of 0.936 on part A and 0.895 on Part B. Our model showed more accurate results for sessile and smaller-sized polyps.

scholarly journals EVALUATION OF SAR TO OPTICAL IMAGE TRANSLATION USING CONDITIONAL GENERATIVE ADVERSARIAL NETWORK FOR CLOUD REMOVAL IN A CROP DATASET

2021 ◽  
Vol XLIII-B3-2021 ◽  
pp. 823-828
Author(s):  
L. E. Christovam ◽  
M. H. Shimabukuro ◽  
M. L. B. T. Galo ◽  
E. Honkavaara
Keyword(s):  
Time Series ◽  
Visual Analysis ◽  
Optical Image ◽  
Optical Data ◽  
Image Pair ◽  
Generative Adversarial Network ◽  
The Real ◽  
Adversarial Network ◽  
Sensing Applications ◽  
Cloud Removal

Abstract. Most methods developed to map crop fields with high-quality are based on optical image time-series. However, often accuracy of these approaches is deteriorated due to clouds and cloud shadows, which can decrease the availably of optical data required to represent crop phenological stages. In this sense, the objective of this study was to implement and evaluate the conditional Generative Adversarial Network (cGAN) that has been indicated as a potential tool to address the cloud and cloud shadow removal; we also compared it with the Witthaker Smother (WS), which is a well-known data cleaning algorithm. The dataset used to train and assess the methods was the Luis Eduardo Magalhães benchmark for tropical agricultural remote sensing applications. We selected one MSI/Sentinel-2 and C-SAR/Sentinel-1 image pair taken in days as close as possible. A total of 5000 image pair patches were generated to train the cGAN model, which was used to derive synthetic optical pixels for a testing area. Visual analysis, spectral behaviour comparison, and classification were used to evaluate and compare the pixels generated with the cGAN and WS against the pixel values from the real image. The cGAN provided consistent pixel values for most crop types in comparison to the real pixel values and outperformed the WS significantly. The results indicated that the cGAN has potential to fill cloud and cloud shadow gaps in optical image time-series.

scholarly journals Panchromatic Image Super-Resolution Via Self Attention-Augmented Wasserstein Generative Adversarial Network

Sensors ◽  
2021 ◽  
Vol 21 (6) ◽  
pp. 2158
Author(s):  
Juan Du ◽  
Kuanhong Cheng ◽  
Yue Yu ◽  
Dabao Wang ◽  
Huixin Zhou
Keyword(s):  
Large Scale ◽  
Spatial Information ◽  
Super Resolution ◽  
Attention Mechanism ◽  
Feature Representation ◽  
Similarity Function ◽  
Feature Maps ◽  
Generative Adversarial Network ◽  
Convolutional Network ◽  
Adversarial Network

Panchromatic (PAN) images contain abundant spatial information that is useful for earth observation, but always suffer from low-resolution ( LR) due to the sensor limitation and large-scale view field. The current super-resolution (SR) methods based on traditional attention mechanism have shown remarkable advantages but remain imperfect to reconstruct the edge details of SR images. To address this problem, an improved SR model which involves the self-attention augmented Wasserstein generative adversarial network ( SAA-WGAN) is designed to dig out the reference information among multiple features for detail enhancement. We use an encoder-decoder network followed by a fully convolutional network (FCN) as the backbone to extract multi-scale features and reconstruct the High-resolution (HR) results. To exploit the relevance between multi-layer feature maps, we first integrate a convolutional block attention module (CBAM) into each skip-connection of the encoder-decoder subnet, generating weighted maps to enhance both channel-wise and spatial-wise feature representation automatically. Besides, considering that the HR results and LR inputs are highly similar in structure, yet cannot be fully reflected in traditional attention mechanism, we, therefore, designed a self augmented attention (SAA) module, where the attention weights are produced dynamically via a similarity function between hidden features; this design allows the network to flexibly adjust the fraction relevance among multi-layer features and keep the long-range inter information, which is helpful to preserve details. In addition, the pixel-wise loss is combined with perceptual and gradient loss to achieve comprehensive supervision. Experiments on benchmark datasets demonstrate that the proposed method outperforms other SR methods in terms of both objective evaluation and visual effect.

scholarly journals 3D MRI Reconstruction Based on 2D Generative Adversarial Network Super-Resolution

Sensors ◽  
2021 ◽  
Vol 21 (9) ◽  
pp. 2978
Author(s):  
Hongtao Zhang ◽  
Yuki Shinomiya ◽  
Shinichi Yoshida
Keyword(s):  
Deep Learning ◽  
Image Quality ◽  
Missing Values ◽  
Brain Mri ◽  
Super Resolution ◽  
Reconstruction Method ◽  
Generative Adversarial Network ◽  
Frequency Information ◽  
Training Costs ◽  
Adversarial Network

The diagnosis of brain pathologies usually involves imaging to analyze the condition of the brain. Magnetic resonance imaging (MRI) technology is widely used in brain disorder diagnosis. The image quality of MRI depends on the magnetostatic field strength and scanning time. Scanners with lower field strengths have the disadvantages of a low resolution and high imaging cost, and scanning takes a long time. The traditional super-resolution reconstruction method based on MRI generally states an optimization problem in terms of prior information. It solves the problem using an iterative approach with a large time cost. Many methods based on deep learning have emerged to replace traditional methods. MRI super-resolution technology based on deep learning can effectively improve MRI resolution through a three-dimensional convolutional neural network; however, the training costs are relatively high. In this paper, we propose the use of two-dimensional super-resolution technology for the super-resolution reconstruction of MRI images. In the first reconstruction, we choose a scale factor of 2 and simulate half the volume of MRI slices as input. We utilize a receiving field block enhanced super-resolution generative adversarial network (RFB-ESRGAN), which is superior to other super-resolution technologies in terms of texture and frequency information. We then rebuild the super-resolution reconstructed slices in the MRI. In the second reconstruction, the image after the first reconstruction is composed of only half of the slices, and there are still missing values. In our previous work, we adopted the traditional interpolation method, and there was still a gap in the visual effect of the reconstructed images. Therefore, we propose a noise-based super-resolution network (nESRGAN). The noise addition to the network can provide additional texture restoration possibilities. We use nESRGAN to further restore MRI resolution and high-frequency information. Finally, we achieve the 3D reconstruction of brain MRI images through two super-resolution reconstructions. Our proposed method is superior to 3D super-resolution technology based on deep learning in terms of perception range and image quality evaluation standards.

scholarly journals VEHICLE DETECTION IN REMOTE SENSING IMAGES USING DEEP NEURAL NETWORKS AND MULTI-TASK LEARNING

2020 ◽  
Vol V-2-2020 ◽  
pp. 797-804
Author(s):  
M. Cao ◽  
H. Ji ◽  
Z. Gao ◽  
T. Mei
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Learning Community ◽  
Feature Fusion ◽  
Super Resolution ◽  
Vehicle Detection ◽  
Remote Sensing Image ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Benchmark Datasets

Abstract. Vehicle detection in remote sensing image has been attracting remarkable attention over past years for its applications in traffic, security, military, and surveillance fields. Due to the stunning success of deep learning techniques in object detection community, we consider to utilize CNNs for vehicle detection task in remote sensing image. Specifically, we take advantage of deep residual network, multi-scale feature fusion, hard example mining and homography augmentation to realize vehicle detection, which almost integrates all the advanced techniques in deep learning community. Furthermore, we simultaneously address super-resolution (SR) and detection problems of low-resolution (LR) image in an end-to-end manner. In consideration of the absence of paired low-/highresolution data which are generally time-consuming and cumbersome to collect, we leverage generative adversarial network (GAN) for unsupervised SR. Detection loss is back-propagated to SR generator to boost detection performance. We conduct experiments on representative benchmark datasets and demonstrate that our model yields significant improvements over state-of-the-art methods in deep learning and remote sensing areas.

Sign in / Sign up

Export Citation Format

Share Document