scholarly journals Generative Adversarial Network-Based Super-Resolution Considering Quantitative and Perceptual Quality

Symmetry ◽  
2020 ◽  
Vol 12 (3) ◽  
pp. 449 ◽  
Author(s):  
Can Li ◽  
Liejun Wang ◽  
Shuli Cheng ◽  
Naixiang Ao
Keyword(s):  
Super Resolution ◽  
Ground Truth ◽  
Second Order ◽  
Attention Mechanism ◽  
Generative Adversarial Networks ◽  
Perceptual Quality ◽  
Human Visual Perception ◽  
Image Perception ◽  
Generative Adversarial Network ◽  
Image Super Resolution

In recent years, the common algorithms for image super-resolution based on deep learning have been increasingly successful, but there is still a large gap between the results generated by each algorithm and the ground-truth. Even some algorithms that are dedicated to image perception produce more textures that do not exist in the original image, and these artefacts also affect the visual perceptual quality of the image. We believe that in the existing perceptual-based image super-resolution algorithm, it is necessary to consider Super-Resolution (SR) image quality, which can restore the important structural parts of the original picture. This paper mainly improves the Enhanced Super-Resolution Generative Adversarial Networks (ESRGAN) algorithm in the following aspects: adding a shallow network structure, adding the dual attention mechanism in the generator and the discriminator, including the second-order channel mechanism and spatial attention mechanism and optimizing perceptual loss by adding second-order covariance normalization at the end of feature extractor. The results of this paper ensure image perceptual quality while reducing image distortion and artefacts, improving the perceived similarity of images and making the images more in line with human visual perception.

scholarly journals TWIST-GAN: Towards Wavelet Transform and Transferred GAN for Spatio-Temporal Single Image Super Resolution

2021 ◽  
Vol 12 (6) ◽  
pp. 1-20
Author(s):  
Fayaz Ali Dharejo ◽  
Farah Deeba ◽  
Yuanchun Zhou ◽  
Bhagwan Das ◽  
Munsif Ali Jatoi ◽  
...  
Keyword(s):  
Remote Sensing ◽  
Super Resolution ◽  
Generative Adversarial Networks ◽  
Single Image ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Image Super Resolution ◽  
Spatio Temporal ◽  
Single Image Super Resolution

Single Image Super-resolution (SISR) produces high-resolution images with fine spatial resolutions from a remotely sensed image with low spatial resolution. Recently, deep learning and generative adversarial networks (GANs) have made breakthroughs for the challenging task of single image super-resolution (SISR) . However, the generated image still suffers from undesirable artifacts such as the absence of texture-feature representation and high-frequency information. We propose a frequency domain-based spatio-temporal remote sensing single image super-resolution technique to reconstruct the HR image combined with generative adversarial networks (GANs) on various frequency bands (TWIST-GAN). We have introduced a new method incorporating Wavelet Transform (WT) characteristics and transferred generative adversarial network. The LR image has been split into various frequency bands by using the WT, whereas the transfer generative adversarial network predicts high-frequency components via a proposed architecture. Finally, the inverse transfer of wavelets produces a reconstructed image with super-resolution. The model is first trained on an external DIV2 K dataset and validated with the UC Merced Landsat remote sensing dataset and Set14 with each image size of 256 × 256. Following that, transferred GANs are used to process spatio-temporal remote sensing images in order to minimize computation cost differences and improve texture information. The findings are compared qualitatively and qualitatively with the current state-of-art approaches. In addition, we saved about 43% of the GPU memory during training and accelerated the execution of our simplified version by eliminating batch normalization layers.

scholarly journals Impact of GAN-based lesion-focused medical image super-resolution on the robustness of radiomic features

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Erick Costa de Farias ◽  
Christian di Noia ◽  
Changhee Han ◽  
Evis Sala ◽  
Mauro Castelli ◽  
...  
Keyword(s):  
Biomarker Discovery ◽  
Super Resolution ◽  
Principal Component ◽  
Medical Decision ◽  
Perceptual Quality ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Proposed Model ◽  
Spatial Pyramid Pooling ◽  
Image Super Resolution

AbstractRobust machine learning models based on radiomic features might allow for accurate diagnosis, prognosis, and medical decision-making. Unfortunately, the lack of standardized radiomic feature extraction has hampered their clinical use. Since the radiomic features tend to be affected by low voxel statistics in regions of interest, increasing the sample size would improve their robustness in clinical studies. Therefore, we propose a Generative Adversarial Network (GAN)-based lesion-focused framework for Computed Tomography (CT) image Super-Resolution (SR); for the lesion (i.e., cancer) patch-focused training, we incorporate Spatial Pyramid Pooling (SPP) into GAN-Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE). At $$2\times $$ 2 × SR, the proposed model achieved better perceptual quality with less blurring than the other considered state-of-the-art SR methods, while producing comparable results at $$4\times $$ 4 × SR. We also evaluated the robustness of our model’s radiomic feature in terms of quantization on a different lung cancer CT dataset using Principal Component Analysis (PCA). Intriguingly, the most important radiomic features in our PCA-based analysis were the most robust features extracted on the GAN-super-resolved images. These achievements pave the way for the application of GAN-based image Super-Resolution techniques for studies of radiomics for robust biomarker discovery.

scholarly journals Edge Loss for Remote Sensing Image Super-Resolution

2021 ◽  
Author(s):  
Jiaoyue Li ◽  
Weifeng Liu ◽  
Kai Zhang ◽  
Baodi Liu
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Super Resolution ◽  
Ground Truth ◽  
Remote Sensing Image ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Sensing Applications ◽  
Image Super Resolution ◽  
Edge Based

Remote sensing image super-resolution (SR) plays an essential role in many remote sensing applications. Recently, remote sensing image super-resolution methods based on deep learning have shown remarkable performance. However, directly utilizing the deep learning methods becomes helpless to recover the remote sensing images with a large number of complex objectives or scene. So we propose an edge-based dense connection generative adversarial network (SREDGAN), which minimizes the edge differences between the generated image and its corresponding ground truth. Experimental results on NWPU-VHR-10 and UCAS-AOD datasets demonstrate that our method improves 1.92 and 0.045 in PSNR and SSIM compared with SRGAN, respectively.

scholarly journals iSeeBetter: Spatio-temporal video super-resolution using recurrent generative back-projection networks

2020 ◽  
Vol 6 (3) ◽  
pp. 307-317
Author(s):  
Aman Chadha ◽  
John Britto ◽  
M. Mani Roja
Keyword(s):  
Mean Squared Error ◽  
Signal To Noise Ratio ◽  
Super Resolution ◽  
Structural Similarity ◽  
Generative Adversarial Networks ◽  
Video Frame ◽  
Perceptual Quality ◽  
Back Projection ◽  
Generative Adversarial Network ◽  
Spatio Temporal

Abstract Recently, learning-based models have enhanced the performance of single-image super-resolution (SISR). However, applying SISR successively to each video frame leads to a lack of temporal coherency. Convolutional neural networks (CNNs) outperform traditional approaches in terms of image quality metrics such as peak signal to noise ratio (PSNR) and structural similarity (SSIM). On the other hand, generative adversarial networks (GANs) offer a competitive advantage by being able to mitigate the issue of a lack of finer texture details, usually seen with CNNs when super-resolving at large upscaling factors. We present iSeeBetter, a novel GAN-based spatio-temporal approach to video super-resolution (VSR) that renders temporally consistent super-resolution videos. iSeeBetter extracts spatial and temporal information from the current and neighboring frames using the concept of recurrent back-projection networks as its generator. Furthermore, to improve the “naturality” of the super-resolved output while eliminating artifacts seen with traditional algorithms, we utilize the discriminator from super-resolution generative adversarial network. Although mean squared error (MSE) as a primary loss-minimization objective improves PSNR/SSIM, these metrics may not capture fine details in the image resulting in misrepresentation of perceptual quality. To address this, we use a four-fold (MSE, perceptual, adversarial, and total-variation loss function. Our results demonstrate that iSeeBetter offers superior VSR fidelity and surpasses state-of-the-art performance.

scholarly journals Devnagari Handwritten Characters Image Super-Resolution based on Enhanced SRGAN

2021 ◽  
Vol 16 (1) ◽  
pp. 103-109
Author(s):  
Prasiddha Siwakoti ◽  
Sharad Kumar Ghimire
Keyword(s):  
Super Resolution ◽  
Image Data ◽  
Ground Truth ◽  
Frequency Component ◽  
High Frequency Component ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Batch Normalization ◽  
Statistical Similarity ◽  
Image Super Resolution

The difficulty in machine learning-based image super-resolution is to generate high-frequency component in an image without introducing any artifacts. In this paper, Devnagari handwritten characters image using a generative adversarial network with a classifier is generated in high-resolution which is also classifiable. The generator architecture is modified by removing all batch normalization layers in generator architecture with a residual in residual dense block. Batch normalization is removed because it produces unwanted artifacts in the generated images. A Devnagari handwritten characters classifier is built using CNN. The classifier is used in the network to calculate the content loss. The adversarial loss is obtained from the GAN architecture and both of the losses are added to obtain total loss. Generated HR images is validated using six different evaluation metrics among which MSE, PSNR determines pixel-wise difference and SSIM compares images perceptually. Similarly, FID is used to measure the statistical similarity between the batch of generated images and its original batch. Finally, the Gradient similarity is used to assess the quality of the generated image. From the experimental results, we obtain MSE, PSNR and SSIM as 0.0507, 12.95(dB) and 0.8172 respectively. Similarly, the FID value obtained was 27.5 with the classification accuracy of image data of 98%. The gradient similarity between the generated image and the ground truth obtained was 0.9124.

Sign in / Sign up

Export Citation Format

Share Document