scholarly journals Generative Adversarial Network-Based Edge-Preserving Superresolution Reconstruction of Infrared Images

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Yuqing Zhao ◽  
Guangyuan Fu ◽  
Hongqiao Wang ◽  
Shaolei Zhang ◽  
Min Yue
Keyword(s):  
Infrared Image ◽  
Reconstruction Method ◽  
Infrared Images ◽  
Generative Adversarial Network ◽  
Edge Preserving ◽  
Edge Structure ◽  
Second Stage ◽  
Adversarial Network ◽  
Objective Quality ◽  
Image Edges

The convolutional neural network has achieved good results in the superresolution reconstruction of single-frame images. However, due to the shortcomings of infrared images such as lack of details, poor contrast, and blurred edges, superresolution reconstruction of infrared images that preserves the edge structure and better visual quality is still challenging. Aiming at the problems of low resolution and unclear edges of infrared images, this work proposes a two-stage generative adversarial network model to reconstruct realistic superresolution images from four times downsampled infrared images. In the first stage of the generative adversarial network, it focuses on recovering the overall contour information of the image to obtain clear image edges; the second stage of the generative adversarial network focuses on recovering the detailed feature information of the image and has a stronger ability to express details. The infrared image superresolution reconstruction method proposed in this work has highly realistic visual effects and good objective quality evaluation results.

Reflection interference removal for infrared thermography images based on GAN

2021 ◽  
Vol 63 (9) ◽  
pp. 529-533
Author(s):  
Jiali Zhang ◽  
Yupeng Tian ◽  
LiPing Ren ◽  
Jiaheng Cheng ◽  
JinChen Shi
Keyword(s):  
Neural Networks ◽  
Infrared Thermography ◽  
Metal Surfaces ◽  
Infrared Image ◽  
Infrared Camera ◽  
Infrared Images ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Continuous Application ◽  
Visible Images

Reflection in images is common and the removal of complex noise such as image reflection is still being explored. The problem is difficult and ill-posed, not only because there is no mixing function but also because there are no constraints in the output space (the processed image). When it comes to detecting defects on metal surfaces using infrared thermography, reflection from smooth metal surfaces can easily affect the final detection results. Therefore, it is essential to remove the reflection interference in infrared images. With the continuous application and expansion of neural networks in the field of image processing, researchers have tried to apply neural networks to remove image reflection. However, they have mainly focused on reflection interference removal in visible images and it is believed that no researchers have applied neural networks to remove reflection interference in infrared images. In this paper, the authors introduce the concept of a conditional generative adversarial network (cGAN) and propose an end-to-end trained network based on this with two types of loss: perceptual loss and adversarial loss. A self-built infrared reflection image dataset from an infrared camera is used. The experimental results demonstrate the effectiveness of this GAN for removing infrared image reflection.

scholarly journals Learning a Generative Model for Fusing Infrared and Visible Images via Conditional Generative Adversarial Network with Dual Discriminators

2019 ◽  
Author(s):  
Han Xu ◽  
Pengwei Liang ◽  
Wei Yu ◽  
Junjun Jiang ◽  
Jiayi Ma
Keyword(s):  
Probability Distribution ◽  
State Of The Art ◽  
Infrared Image ◽  
Infrared Images ◽  
Generative Adversarial Network ◽  
Visible Image ◽  
Qualitative And Quantitative ◽  
Adversarial Network ◽  
Fused Image ◽  
Visible Images

In this paper, we propose a new end-to-end model, called dual-discriminator conditional generative adversarial network (DDcGAN), for fusing infrared and visible images of different resolutions. Unlike the pixel-level methods and existing deep learning-based methods, the fusion task is accomplished through the adversarial process between a generator and two discriminators, in addition to the specially designed content loss. The generator is trained to generate real-like fused images to fool discriminators. The two discriminators are trained to calculate the JS divergence between the probability distribution of downsampled fused images and infrared images, and the JS divergence between the probability distribution of gradients of fused images and gradients of visible images, respectively. Thus, the fused images can compensate for the features that are not constrained by the single content loss. Consequently, the prominence of thermal targets in the infrared image and the texture details in the visible image can be preserved or even enhanced in the fused image simultaneously. Moreover, by constraining and distinguishing between the downsampled fused image and the low-resolution infrared image, DDcGAN can be preferably applied to the fusion of different resolution images. Qualitative and quantitative experiments on publicly available datasets demonstrate the superiority of our method over the state-of-the-art.

scholarly journals Patch-Wise Infrared and Visible Image Fusion Using Spatial Adaptive Weights

2021 ◽  
Vol 11 (19) ◽  
pp. 9255
Author(s):  
Syeda Minahil ◽  
Jun-Hyung Kim ◽  
Youngbae Hwang
Keyword(s):  
Image Fusion ◽  
Infrared Image ◽  
Source Image ◽  
Infrared Images ◽  
Generative Adversarial Network ◽  
Visible Image ◽  
Significant Information ◽  
Adaptive Weights ◽  
Adversarial Network ◽  
Fusion Methods

In infrared (IR) and visible image fusion, the significant information is extracted from each source image and integrated into a single image with comprehensive data. We observe that the salient regions in the infrared image contain targets of interests. Therefore, we enforce spatial adaptive weights derived from the infrared images. In this paper, a Generative Adversarial Network (GAN)-based fusion method is proposed for infrared and visible image fusion. Based on the end-to-end network structure with dual discriminators, a patch-wise discrimination is applied to reduce blurry artifact from the previous image-level approaches. A new loss function is also proposed to use constructed weight maps which direct the adversarial training of GAN in a manner such that the informative regions of the infrared images are preserved. Experiments are performed on the two datasets and ablation studies are also conducted. The qualitative and quantitative analysis shows that we achieve competitive results compared to the existing fusion methods.

scholarly journals Classification of Space Objects by Using Deep Learning with Micro-Doppler Signature Images

Sensors ◽  
2021 ◽  
Vol 21 (13) ◽  
pp. 4365
Author(s):  
Kwangyong Jung ◽  
Jae-In Lee ◽  
Nammoon Kim ◽  
Sunjin Oh ◽  
Dong-Wook Seo
Keyword(s):  
Transfer Learning ◽  
Data Augmentation ◽  
Doppler Frequency ◽  
Extraction Methods ◽  
Classification Performance ◽  
Reconstruction Method ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Space Objects ◽  
Target Characteristics

Radar target classification is an important task in the missile defense system. State-of-the-art studies using micro-doppler frequency have been conducted to classify the space object targets. However, existing studies rely highly on feature extraction methods. Therefore, the generalization performance of the classifier is limited and there is room for improvement. Recently, to improve the classification performance, the popular approaches are to build a convolutional neural network (CNN) architecture with the help of transfer learning and use the generative adversarial network (GAN) to increase the training datasets. However, these methods still have drawbacks. First, they use only one feature to train the network. Therefore, the existing methods cannot guarantee that the classifier learns more robust target characteristics. Second, it is difficult to obtain large amounts of data that accurately mimic real-world target features by performing data augmentation via GAN instead of simulation. To mitigate the above problem, we propose a transfer learning-based parallel network with the spectrogram and the cadence velocity diagram (CVD) as the inputs. In addition, we obtain an EM simulation-based dataset. The radar-received signal is simulated according to a variety of dynamics using the concept of shooting and bouncing rays with relative aspect angles rather than the scattering center reconstruction method. Our proposed model is evaluated on our generated dataset. The proposed method achieved about 0.01 to 0.39% higher accuracy than the pre-trained networks with a single input feature.

scholarly journals Promising Generative Adversarial Network Based Sinogram Inpainting Method for Ultra-Limited-Angle Computed Tomography Imaging

Sensors ◽  
2019 ◽  
Vol 19 (18) ◽  
pp. 3941 ◽  
Author(s):  
Li ◽  
Cai ◽  
Wang ◽  
Zhang ◽  
Tang ◽  
...  
Keyword(s):  
Computed Tomography ◽  
Imaging System ◽  
Reconstruction Method ◽  
Ct Reconstruction ◽  
Reconstruction Algorithms ◽  
Generative Adversarial Network ◽  
Image Domain ◽  
Adversarial Network ◽  
Domain Loss ◽  
Limited Angle

Limited-angle computed tomography (CT) image reconstruction is a challenging problem in the field of CT imaging. In some special applications, limited by the geometric space and mechanical structure of the imaging system, projections can only be collected with a scanning range of less than 90°. We call this kind of serious limited-angle problem the ultra-limited-angle problem, which is difficult to effectively alleviate by traditional iterative reconstruction algorithms. With the development of deep learning, the generative adversarial network (GAN) performs well in image inpainting tasks and can add effective image information to restore missing parts of an image. In this study, given the characteristic of GAN to generate missing information, the sinogram-inpainting-GAN (SI-GAN) is proposed to restore missing sinogram data to suppress the singularity of the truncated sinogram for ultra-limited-angle reconstruction. We propose the U-Net generator and patch-design discriminator in SI-GAN to make the network suitable for standard medical CT images. Furthermore, we propose a joint projection domain and image domain loss function, in which the weighted image domain loss can be added by the back-projection operation. Then, by inputting a paired limited-angle/180° sinogram into the network for training, we can obtain the trained model, which has extracted the continuity feature of sinogram data. Finally, the classic CT reconstruction method is used to reconstruct the images after obtaining the estimated sinograms. The simulation studies and actual data experiments indicate that the proposed method performed well to reduce the serious artifacts caused by ultra-limited-angle scanning.

scholarly journals Generative Adversarial Networks for Visible to Infrared Video Conversion

2020 ◽  
Author(s):  
Mohammad Shahab Uddin ◽  
Jiang Li
Keyword(s):  
Deep Learning ◽  
Performance Metrics ◽  
Infrared Image ◽  
Image Databases ◽  
Learning Models ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Optical Images ◽  
Infrared Video ◽  
Image Pairs

Deep learning models are data driven. For example, the most popular convolutional neural network (CNN) model used for image classification or object detection requires large labeled databases for training to achieve competitive performances. This requirement is not difficult to be satisfied in the visible domain since there are lots of labeled video and image databases available nowadays. However, given the less popularity of infrared (IR) camera, the availability of labeled infrared videos or image databases is limited. Therefore, training deep learning models in infrared domain is still challenging. In this chapter, we applied the pix2pix generative adversarial network (Pix2Pix GAN) and cycle-consistent GAN (Cycle GAN) models to convert visible videos to infrared videos. The Pix2Pix GAN model requires visible-infrared image pairs for training while the Cycle GAN relaxes this constraint and requires only unpaired images from both domains. We applied the two models to an open-source database where visible and infrared videos provided by the signal multimedia and telecommunications laboratory at the Federal University of Rio de Janeiro. We evaluated conversion results by performance metrics including Inception Score (IS), Frechet Inception Distance (FID) and Kernel Inception Distance (KID). Our experiments suggest that cycle-consistent GAN is more effective than pix2pix GAN for generating IR images from optical images.

scholarly journals Deep Representation of a Normal Map for Screen-Space Fluid Rendering

2021 ◽  
Vol 11 (19) ◽  
pp. 9065
Author(s):  
Myungjin Choi ◽  
Jee-Hyeok Park ◽  
Qimeng Zhang ◽  
Byeung-Sun Hong ◽  
Chang-Hun Kim
Keyword(s):  
Surface Reconstruction ◽  
Computation Time ◽  
Fluid Simulation ◽  
Image Resolution ◽  
Reconstruction Method ◽  
Generative Adversarial Network ◽  
Normal Map ◽  
Adversarial Network ◽  
Reconstruction Methods ◽  
Screen Space

We propose a novel method for addressing the problem of efficiently generating a highly refined normal map for screen-space fluid rendering. Because the process of filtering the normal map is crucially important to ensure the quality of the final screen-space fluid rendering, we employ a conditional generative adversarial network (cGAN) as a filter that learns a deep normal map representation, thereby refining the low-quality normal map. In particular, we have designed a novel loss function dedicated to refining the normal map information, and we use a specific set of auxiliary features to train the cGAN generator to learn features that are more robust with respect to edge details. Additionally, we constructed a dataset of six different typical scenes to enable effective demonstrations of multitype fluid simulation. Experiments indicated that our generator was able to infer clearer and more detailed features for this dataset than a basic screen-space fluid rendering method. Moreover, in some cases, the results generated by our method were even smoother than those generated by the conventional surface reconstruction method. Our method improves the fluid rendering results via the high-quality normal map while preserving the advantages of the screen-space fluid rendering methods and the traditional surface reconstruction methods, including that of the computation time being independent of the number of simulation particles and the spatial resolution being related only to image resolution.

Sign in / Sign up

Export Citation Format

Share Document