scholarly journals SharpGAN: Dynamic Scene Deblurring Method for Smart Ship Based on Receptive Field Block and Generative Adversarial Networks

Sensors ◽  
2021 ◽  
Vol 21 (11) ◽  
pp. 3641
Author(s):  
Hui Feng ◽  
Jundong Guo ◽  
Haixiang Xu ◽  
Shuzhi Sam Ge
Keyword(s):  
Receptive Field ◽  
Evaluation Criteria ◽  
Objective Evaluation ◽  
Detection Algorithm ◽  
Image Features ◽  
Motion Blur ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Blurred Image

Complex marine environment has an adverse effect on the object detection algorithm based on the vision sensor for the smart ship sailing at sea. In order to eliminate the motion blur in the images during the navigation of the smart ship and ensure safety, we propose SharpGAN, a new image deblurring method based on the generative adversarial network (GAN). First of all, we introduce the receptive field block net (RFBNet) to the deblurring network to enhance the network’s ability to extract blurred image features. Secondly, we propose a feature loss that combines different levels of image features to guide the network to perform higher-quality deblurring and improve the feature similarity between the restored images and the sharp images. Besides, we use the lightweight RFB-s module to significantly improve the real-time performance of the deblurring network. Compared with the existing deblurring methods, the proposed method not only has better deblurring performance in subjective visual effects and objective evaluation criteria, but also has higher deblurring efficiency. Finally, the experimental results reveal that the SharpGAN has a high correlation with the deblurring methods based on the physical model.

scholarly journals Lightweight End-to-End Speech Enhancement Generative Adversarial Network Using Sinc Convolutions

2021 ◽  
Vol 11 (16) ◽  
pp. 7564
Author(s):  
Lujun Li ◽  
Wudamu ◽  
Ludwig Kürzinger ◽  
Tobias Watzel ◽  
Gerhard Rigoll
Keyword(s):  
Speech Enhancement ◽  
Data Augmentation ◽  
Evaluation Criteria ◽  
Objective Evaluation ◽  
Research Direction ◽  
Generative Adversarial Networks ◽  
Speech Signal Processing ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
End To End

Generative adversarial networks (GANs) have recently garnered significant attention for their use in speech enhancement tasks, in which they generally process and reconstruct speech waveforms directly. Existing GANs for speech enhancement rely solely on the convolution operation, which may not accurately characterize the local information of speech signals—particularly high-frequency components. Sinc convolution has been proposed in order to allow the GAN to learn more meaningful filters in the input layer, and has achieved remarkable success in several speech signal processing tasks. Nevertheless, Sinc convolution for speech enhancement is still an under-explored research direction. This paper proposes Sinc–SEGAN, a novel generative adversarial architecture for speech enhancement, which usefully merges two powerful paradigms: Sinc convolution and the speech enhancement GAN (SEGAN). There are two highlights of the proposed system. First, it works in an end-to-end manner, overcoming the distortion caused by imperfect phase estimation. Second, the system derives a customized filter bank, tuned for the desired application compactly and efficiently. We empirically study the influence of different configurations of Sinc convolution, including the placement of the Sinc convolution layer, length of input signals, number of Sinc filters, and kernel size of Sinc convolution. Moreover, we employ a set of data augmentation techniques in the time domain, which further improve the system performance and its generalization abilities. Compared to competitive baseline systems, Sinc–SEGAN overtakes all of them with drastically reduced system parameters, demonstrating its effectiveness for practical usage, e.g., hearing aid design and cochlear implants. Additionally, data augmentation methods further boost Sinc–SEGAN performance across classic objective evaluation criteria for speech enhancement.

Colorization of fusion image of infrared and visible images based on parallel generative adversarial network approach

2021 ◽  
pp. 1-10
Author(s):  
Lei Chen ◽  
Jun Han ◽  
Feng Tian
Keyword(s):  
Ground Truth ◽  
Objective Evaluation ◽  
Generative Adversarial Networks ◽  
Data Sets ◽  
Fusion Image ◽  
Generative Adversarial Network ◽  
Visible Image ◽  
Low Contrast ◽  
Adversarial Network ◽  
Visible Images

Fusing the infrared (IR) and visible images has many advantages and can be applied to applications such as target detection and recognition. Colors can give more accurate and distinct features, but the low resolution and low contrast of fused images make this a challenge task. In this paper, we proposed a method based on parallel generative adversarial networks (GANs) to address the challenge. We used IR image, visible image and fusion image as ground truth of ‘L’, ‘a’ and ‘b’ of the Lab model. Through the parallel GANs, we can gain the Lab data which can be converted to RGB image. We adopt TNO and RoadScene data sets to verify our method, and compare with five objective evaluation parameters obtained by other three methods based on deep learning (DL). It is demonstrated that the proposed approach is able to achieve better performance against state-of-arts methods.

scholarly journals Super-Resolution of Remote Sensing Images via a Dense Residual Generative Adversarial Network

2019 ◽  
Vol 11 (21) ◽  
pp. 2578 ◽  
Author(s):  
Wen Ma ◽  
Zongxu Pan ◽  
Feng Yuan ◽  
Bin Lei
Keyword(s):  
Remote Sensing ◽  
Network Architecture ◽  
Super Resolution ◽  
Objective Evaluation ◽  
Generative Adversarial Networks ◽  
Remote Sensing Images ◽  
Generative Adversarial Network ◽  
Memory Mechanism ◽  
Adversarial Network ◽  
Sensing Applications

Single image super-resolution (SISR) has been widely studied in recent years as a crucial technique for remote sensing applications. In this paper, a dense residual generative adversarial network (DRGAN)-based SISR method is proposed to promote the resolution of remote sensing images. Different from previous super-resolution (SR) approaches based on generative adversarial networks (GANs), the novelty of our method mainly lies in the following factors. First, we made a breakthrough in terms of network architecture to improve performance. We designed a dense residual network as the generative network in GAN, which can make full use of the hierarchical features from low-resolution (LR) images. We also introduced a contiguous memory mechanism into the network to take advantage of the dense residual block. Second, we modified the loss function and altered the model of the discriminative network according to the Wasserstein GAN with a gradient penalty (WGAN-GP) for stable training. Extensive experiments were performed using the NWPU-RESISC45 dataset, and the results demonstrated that the proposed method outperforms state-of-the-art methods in terms of both objective evaluation and subjective perspective.

Fine-Grained Semantic Image Synthesis with Object-Attention Generative Adversarial Network

2021 ◽  
Vol 12 (5) ◽  
pp. 1-18
Author(s):  
Min Wang ◽  
Congyan Lang ◽  
Liqian Liang ◽  
Songhe Feng ◽  
Tao Wang ◽  
...  
Keyword(s):  
Feature Fusion ◽  
Image Synthesis ◽  
Image Features ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Fine Grained ◽  
Multi Scale ◽  
Adversarial Network ◽  
Object Attention ◽  
Object Features

Semantic image synthesis is a new rising and challenging vision problem accompanied by the recent promising advances in generative adversarial networks. The existing semantic image synthesis methods only consider the global information provided by the semantic segmentation mask, such as class label, global layout, and location, so the generative models cannot capture the rich local fine-grained information of the images (e.g., object structure, contour, and texture). To address this issue, we adopt a multi-scale feature fusion algorithm to refine the generated images by learning the fine-grained information of the local objects. We propose OA-GAN, a novel object-attention generative adversarial network that allows attention-driven, multi-fusion refinement for fine-grained semantic image synthesis. Specifically, the proposed model first generates multi-scale global image features and local object features, respectively, then the local object features are fused into the global image features to improve the correlation between the local and the global. In the process of feature fusion, the global image features and the local object features are fused through the channel-spatial-wise fusion block to learn ‘what’ and ‘where’ to attend in the channel and spatial axes, respectively. The fused features are used to construct correlation filters to obtain feature response maps to determine the locations, contours, and textures of the objects. Extensive quantitative and qualitative experiments on COCO-Stuff, ADE20K and Cityscapes datasets demonstrate that our OA-GAN significantly outperforms the state-of-the-art methods.

ORGANIC (1).pdf

2017 ◽  
Author(s):  
Benjamin Sanchez-Lengeling ◽  
Carlos Outeiral ◽  
Gabriel L. Guimaraes ◽  
Alan Aspuru-Guzik
Keyword(s):  
Machine Learning ◽  
Learning Community ◽  
Chemical Species ◽  
Material Design ◽  
Organic Photovoltaic ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Photovoltaic Material

Molecular discovery seeks to generate chemical species tailored to very specific needs. In this paper, we present ORGANIC, a framework based on Objective-Reinforced Generative Adversarial Networks (ORGAN), capable of producing a distribution over molecular space that matches with a certain set of desirable metrics. This methodology combines two successful techniques from the machine learning community: a Generative Adversarial Network (GAN), to create non-repetitive sensible molecular species, and Reinforcement Learning (RL), to bias this generative distribution towards certain attributes. We explore several applications, from optimization of random physicochemical properties to candidates for drug discovery and organic photovoltaic material design.

scholarly journals Restoring Raindrops Using Attentive Generative Adversarial Networks

2021 ◽  
Vol 11 (15) ◽  
pp. 7034
Author(s):  
Hee-Deok Yang
Keyword(s):  
Weather Conditions ◽  
Recurrent Network ◽  
Generative Adversarial Networks ◽  
Navigation Systems ◽  
Vision Systems ◽  
Generative Adversarial Network ◽  
Network Layers ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Outdoor Vision

Artificial intelligence technologies and vision systems are used in various devices, such as automotive navigation systems, object-tracking systems, and intelligent closed-circuit televisions. In particular, outdoor vision systems have been applied across numerous fields of analysis. Despite their widespread use, current systems work well under good weather conditions. They cannot account for inclement conditions, such as rain, fog, mist, and snow. Images captured under inclement conditions degrade the performance of vision systems. Vision systems need to detect, recognize, and remove noise because of rain, snow, and mist to boost the performance of the algorithms employed in image processing. Several studies have targeted the removal of noise resulting from inclement conditions. We focused on eliminating the effects of raindrops on images captured with outdoor vision systems in which the camera was exposed to rain. An attentive generative adversarial network (ATTGAN) was used to remove raindrops from the images. This network was composed of two parts: an attentive-recurrent network and a contextual autoencoder. The ATTGAN generated an attention map to detect rain droplets. A de-rained image was generated by increasing the number of attentive-recurrent network layers. We increased the number of visual attentive-recurrent network layers in order to prevent gradient sparsity so that the entire generation was more stable against the network without preventing the network from converging. The experimental results confirmed that the extended ATTGAN could effectively remove various types of raindrops from images.

scholarly journals Prediction and analysis of multiple protein lysine modified sites based on conditional wasserstein generative adversarial networks

2021 ◽  
Vol 22 (1) ◽  
Author(s):  
Yingxi Yang ◽  
Hui Wang ◽  
Wen Li ◽  
Xiaobo Wang ◽  
Shizhao Wei ◽  
...  
Keyword(s):  
Correlation Coefficient ◽  
Sequence Data ◽  
Rapid Development ◽  
Pearson Correlation ◽  
Structural Features ◽  
Generative Adversarial Networks ◽  
Post Translational Modification ◽  
Generative Adversarial Network ◽  
Data Imbalance ◽  
Adversarial Network

Abstract Background Protein post-translational modification (PTM) is a key issue to investigate the mechanism of protein’s function. With the rapid development of proteomics technology, a large amount of protein sequence data has been generated, which highlights the importance of the in-depth study and analysis of PTMs in proteins. Method We proposed a new multi-classification machine learning pipeline MultiLyGAN to identity seven types of lysine modified sites. Using eight different sequential and five structural construction methods, 1497 valid features were remained after the filtering by Pearson correlation coefficient. To solve the data imbalance problem, Conditional Generative Adversarial Network (CGAN) and Conditional Wasserstein Generative Adversarial Network (CWGAN), two influential deep generative methods were leveraged and compared to generate new samples for the types with fewer samples. Finally, random forest algorithm was utilized to predict seven categories. Results In the tenfold cross-validation, accuracy (Acc) and Matthews correlation coefficient (MCC) were 0.8589 and 0.8376, respectively. In the independent test, Acc and MCC were 0.8549 and 0.8330, respectively. The results indicated that CWGAN better solved the existing data imbalance and stabilized the training error. Alternatively, an accumulated feature importance analysis reported that CKSAAP, PWM and structural features were the three most important feature-encoding schemes. MultiLyGAN can be found at https://github.com/Lab-Xu/MultiLyGAN. Conclusions The CWGAN greatly improved the predictive performance in all experiments. Features derived from CKSAAP, PWM and structure schemes are the most informative and had the greatest contribution to the prediction of PTM.

scholarly journals Stochastic Restoration of Heavily Compressed Musical Audio Using Generative Adversarial Networks

Electronics ◽  
2021 ◽  
Vol 10 (11) ◽  
pp. 1349
Author(s):  
Stefan Lattner ◽  
Javier Nistal
Keyword(s):  
Data Storage ◽  
Audio Signal ◽  
Human Perception ◽  
Generative Adversarial Networks ◽  
Audio Signals ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Extensive Evaluation ◽  
Listening Tests ◽  
Musical Audio

Lossy audio codecs compress (and decompress) digital audio streams by removing information that tends to be inaudible in human perception. Under high compression rates, such codecs may introduce a variety of impairments in the audio signal. Many works have tackled the problem of audio enhancement and compression artifact removal using deep-learning techniques. However, only a few works tackle the restoration of heavily compressed audio signals in the musical domain. In such a scenario, there is no unique solution for the restoration of the original signal. Therefore, in this study, we test a stochastic generator of a Generative Adversarial Network (GAN) architecture for this task. Such a stochastic generator, conditioned on highly compressed musical audio signals, could one day generate outputs indistinguishable from high-quality releases. Therefore, the present study may yield insights into more efficient musical data storage and transmission. We train stochastic and deterministic generators on MP3-compressed audio signals with 16, 32, and 64 kbit/s. We perform an extensive evaluation of the different experiments utilizing objective metrics and listening tests. We find that the models can improve the quality of the audio signals over the MP3 versions for 16 and 32 kbit/s and that the stochastic generators are capable of generating outputs that are closer to the original signals than those of the deterministic generators.

scholarly journals Remote Sensing Image Dataset Expansion Based on Generative Adversarial Networks with Modified Shuffle Attention

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4867
Author(s):  
Lu Chen ◽  
Hongjun Wang ◽  
Xianghao Meng
Keyword(s):  
Remote Sensing ◽  
Neural Networks ◽  
Image Processing ◽  
Remote Sensing Image ◽  
Generative Adversarial Networks ◽  
Generative Adversarial Network ◽  
Evaluation Indexes ◽  
Adversarial Network ◽  
Remote Sensing Image Processing ◽  
Data Expansion

With the development of science and technology, neural networks, as an effective tool in image processing, play an important role in gradual remote-sensing image-processing. However, the training of neural networks requires a large sample database. Therefore, expanding datasets with limited samples has gradually become a research hotspot. The emergence of the generative adversarial network (GAN) provides new ideas for data expansion. Traditional GANs either require a large number of input data, or lack detail in the pictures generated. In this paper, we modify a shuffle attention network and introduce it into GAN to generate higher quality pictures with limited inputs. In addition, we improved the existing resize method and proposed an equal stretch resize method to solve the problem of image distortion caused by different input sizes. In the experiment, we also embed the newly proposed coordinate attention (CA) module into the backbone network as a control test. Qualitative indexes and six quantitative evaluation indexes were used to evaluate the experimental results, which show that, compared with other GANs used for picture generation, the modified Shuffle Attention GAN proposed in this paper can generate more refined and high-quality diversified aircraft pictures with more detailed features of the object under limited datasets.

scholarly journals Enhanced network optimized generative adversarial network for image enhancement

2021 ◽  
Author(s):  
Lingyu Yan ◽  
Jiarun Fu ◽  
Chunzhi Wang ◽  
Zhiwei Ye ◽  
Hongwei Chen ◽  
...  
Keyword(s):  
Image Enhancement ◽  
Image Recognition ◽  
Generative Adversarial Networks ◽  
Low Light ◽  
Generative Adversarial Network ◽  
Adversarial Network ◽  
Adversarial Networks ◽  
Enhancement Method ◽  
New Space ◽  
Traditional Image

AbstractWith the development of image recognition technology, face, body shape, and other factors have been widely used as identification labels, which provide a lot of convenience for our daily life. However, image recognition has much higher requirements for image conditions than traditional identification methods like a password. Therefore, image enhancement plays an important role in the process of image analysis for images with noise, among which the image of low-light is the top priority of our research. In this paper, a low-light image enhancement method based on the enhanced network module optimized Generative Adversarial Networks(GAN) is proposed. The proposed method first applied the enhancement network to input the image into the generator to generate a similar image in the new space, Then constructed a loss function and minimized it to train the discriminator, which is used to compare the image generated by the generator with the real image. We implemented the proposed method on two image datasets (DPED, LOL), and compared it with both the traditional image enhancement method and the deep learning approach. Experiments showed that our proposed network enhanced images have higher PNSR and SSIM, the overall perception of relatively good quality, demonstrating the effectiveness of the method in the aspect of low illumination image enhancement.

Sign in / Sign up

Export Citation Format

Share Document