scholarly journals Virtual View Generation Based on 3D-Dense-Attentive GAN Networks

Sensors ◽  
2019 ◽  
Vol 19 (2) ◽  
pp. 344 ◽  
Author(s):  
Junwei Fu ◽  
Jun Liang
Keyword(s):  
Binocular Vision ◽  
Vision System ◽  
Depth Map ◽  
Image Features ◽  
Generative Adversarial Networks ◽  
Adversarial Networks ◽  
Virtual View ◽  
Proposed Model ◽  
Image Edge ◽  
3D Cnn

A binocular vision system is a common perception component of an intelligent vehicle. Benefiting from the biomimetic structure, the system is simple and effective. Which are extremely snesitive on external factors, especially missing vision signals. In this paper, a virtual view-generation algorithm based on generative adversarial networks (GAN) is proposed to enhance the robustness of binocular vision systems. The proposed model consists of two parts: generative network and discriminator network. To improve the quality of a virtual view, a generative network structure based on 3D convolutional neural networks (3D-CNN) and attentive mechanisms is introduced to extract the time-series features from image sequences. To avoid gradient vanish during training, the dense block structure is utilized to improve the discriminator network. Meanwhile, three kinds of image features, including image edge, depth map and optical flow are extracted to constrain the supervised training of model. The final results on KITTI and Cityscapes datasets demonstrate that our algorithm outperforms conventional methods, and the missing vision signal can be replaced by a generated virtual view.

scholarly journals Conditional Deep 3D-Convolutional Generative Adversarial Nets for RGB-D Generation

2021 ◽  
Vol 2021 ◽  
pp. 1-8
Author(s):  
Richa Sharma ◽  
Manoj Sharma ◽  
Ankit Shukla ◽  
Santanu Chaudhury
Keyword(s):  
Object Tracking ◽  
Action Recognition ◽  
Synthetic Data ◽  
Depth Map ◽  
Generative Adversarial Networks ◽  
Data Generation ◽  
Adversarial Networks ◽  
Proposed Model ◽  
Spatio Temporal ◽  
Class Labels

Generation of synthetic data is a challenging task. There are only a few significant works on RGB video generation and no pertinent works on RGB-D data generation. In the present work, we focus our attention on synthesizing RGB-D data which can further be used as dataset for various applications like object tracking, gesture recognition, and action recognition. This paper has put forward a proposal for a novel architecture that uses conditional deep 3D-convolutional generative adversarial networks to synthesize RGB-D data by exploiting 3D spatio-temporal convolutional framework. The proposed architecture can be used to generate virtually unlimited data. In this work, we have presented the architecture to generate RGB-D data conditioned on class labels. In the architecture, two parallel paths were used, one to generate RGB data and the second to synthesize depth map. The output from the two parallel paths is combined to generate RGB-D data. The proposed model is used for video generation at 30 fps (frames per second). The frame referred here is an RGB-D with the spatial resolution of 512 × 512.

scholarly journals Multi-Turn Chatbot Based on Query-Context Attentions and Dual Wasserstein Generative Adversarial Networks

2019 ◽  
Vol 9 (18) ◽  
pp. 3908 ◽  
Author(s):  
Jintae Kim ◽  
Shinhyeok Oh ◽  
Oh-Woog Kwon ◽  
Harksoo Kim
Keyword(s):  
Performance Measures ◽  
State Of The Art ◽  
Attention Mechanism ◽  
Generative Adversarial Networks ◽  
Training Method ◽  
Adversarial Networks ◽  
Proposed Model ◽  
Previous State ◽  
Vector Representations

To generate proper responses to user queries, multi-turn chatbot models should selectively consider dialogue histories. However, previous chatbot models have simply concatenated or averaged vector representations of all previous utterances without considering contextual importance. To mitigate this problem, we propose a multi-turn chatbot model in which previous utterances participate in response generation using different weights. The proposed model calculates the contextual importance of previous utterances by using an attention mechanism. In addition, we propose a training method that uses two types of Wasserstein generative adversarial networks to improve the quality of responses. In experiments with the DailyDialog dataset, the proposed model outperformed the previous state-of-the-art models based on various performance measures.

scholarly journals Parallel Image Completion with Edge and Color Map

2019 ◽  
Vol 9 (18) ◽  
pp. 3856 ◽  
Author(s):  
Dan Zhao ◽  
Baolong Guo ◽  
Yunyi Yan
Keyword(s):  
Superior Performance ◽  
Generative Adversarial Networks ◽  
Image Completion ◽  
Significant Progress ◽  
Color Information ◽  
Parallel Edge ◽  
Adversarial Networks ◽  
Parallel Image ◽  
Latent Space ◽  
Image Edge

Over the last few years, image completion has made significant progress due to the generative adversarial networks (GANs) that are able to synthesize photorealistic contents. However, one of the main obstacles faced by many existing methods is that they often create blurry textures or distorted structures that are inconsistent with surrounding regions. The main reason is the ineffectiveness of disentangling style latent space implicitly from images. To address this problem, we develop a novel image completion framework called PIC-EC: parallel image completion networks with edge and color maps, which explicitly provides image edge and color information as the prior knowledge for image completion. The PIC-EC framework consists of the parallel edge and color generators followed by an image completion network. Specifically, the parallel paths generate edge and color maps for the missing region at the same time, and then the image completion network fills the missing region with fine details using the generated edge and color information as the priors. The proposed method was evaluated over CelebA-HQ and Paris StreetView datasets. Experimental results demonstrate that PIC-EC achieves superior performance on challenging cases with complex compositions and outperforms existing methods on evaluations of realism and accuracy, both quantitatively and qualitatively.

scholarly journals Multi-Attribute Transfer via Disentangled Representation

2019 ◽  
Vol 33 ◽  
pp. 9195-9202 ◽  
Author(s):  
Jianfu Zhang ◽  
Yuanyuan Huang ◽  
Yaoyi Li ◽  
Weijie Zhao ◽  
Liqing Zhang
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Generative Adversarial Networks ◽  
Significant Progress ◽  
Target Domain ◽  
Adversarial Networks ◽  
Proposed Model ◽  
Image Translation ◽  
Realistic Images ◽  
Novel Model

Recent studies show significant progress in image-to-image translation task, especially facilitated by Generative Adversarial Networks. They can synthesize highly realistic images and alter the attribute labels for the images. However, these works employ attribute vectors to specify the target domain which diminishes image-level attribute diversity. In this paper, we propose a novel model formulating disentangled representations by projecting images to latent units, grouped feature channels of Convolutional Neural Network, to disassemble the information between different attributes. Thanks to disentangled representation, we can transfer attributes according to the attribute labels and moreover retain the diversity beyond the labels, namely, the styles inside each image. This is achieved by specifying some attributes and swapping the corresponding latent units to “swap” the attributes appearance, or applying channel-wise interpolation to blend different attributes. To verify the motivation of our proposed model, we train and evaluate our model on face dataset CelebA. Furthermore, the evaluation of another facial expression dataset RaFD demonstrates the generalizability of our proposed model.

scholarly journals Cell Morphology-Guided De Novo Hit Design by Conditioning Generative Adversarial Networks on Phenotypic Image Features

2020 ◽  
Author(s):  
Oscar Méndez-Lucio ◽  
Paula Andrea Marin Zapata ◽  
Joerg Wichard ◽  
David Rouquié ◽  
Djork-Arné Clevert
Keyword(s):  
Small Molecules ◽  
De Novo ◽  
Molecular Targets ◽  
De Novo Design ◽  
Image Features ◽  
Mitigation Strategy ◽  
Generative Adversarial Networks ◽  
Target Information ◽  
Adversarial Networks ◽  
First Time

Developing new small molecules that are bioactive is time-consuming, costly and rarely successful. As a mitigation strategy, we apply, for the first time, generative adversarial networks to de novo design of small molecules using a phenotype-based drug discovery approach. We trained our model on a set of 30,000 compounds and their respective morphological profiles extracted from high content images; no target information was used to train the model. Using this approach, we were able to automatically design agonist-like compounds of different molecular targets.

scholarly journals Combining Variational Autoencoders & Generative Adversarial Networks to Improve Image Quality

2019 ◽  
Author(s):  
Atin Sakkeer Hussain
Keyword(s):  
Image Quality ◽  
Random Noise ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
Improve Image Quality ◽  
Adversarial Networks ◽  
Proposed Model ◽  
Variational Autoencoder ◽  
Proper Training ◽  
Better Than

Generative Adversarial Networks(GAN) are trained to generate images from random noise vectors, but often these images turn out poorly due to any of several reasons such as model collapse, lack of proper training data, lack of training, etc. To combat this issue this paper, makes use of a Variational Autoencoder(VAE). The VAE is trained on a combination of the training & generated data, after this the VAE can be used to map images generated by the GAN to better versions of it. (This is similar to Denoising, but with few variations in the image). In addition to improving quality the proposed model is shown to work better than normal WGAN’s on sparse datasets with higher variety, in equal number of training epochs.

scholarly journals Various Generative Adversarial Networks Model for Synthetic Prohibitory Sign Image Generation

2021 ◽  
Vol 11 (7) ◽  
pp. 2913
Author(s):  
Christine Dewi ◽  
Rung-Ching Chen ◽  
Yan-Ting Liu ◽  
Hui Yu
Keyword(s):  
Mean Squared Error ◽  
Vision System ◽  
Similarity Index ◽  
Training Data ◽  
Generative Adversarial Networks ◽  
Synthetic Image ◽  
Real Image ◽  
Traffic Sign ◽  
Research Issues ◽  
Adversarial Networks

A synthetic image is a critical issue for computer vision. Traffic sign images synthesized from standard models are commonly used to build computer recognition algorithms for acquiring more knowledge on various and low-cost research issues. Convolutional Neural Network (CNN) achieves excellent detection and recognition of traffic signs with sufficient annotated training data. The consistency of the entire vision system is dependent on neural networks. However, locating traffic sign datasets from most countries in the world is complicated. This work uses various generative adversarial networks (GAN) models to construct intricate images, such as Least Squares Generative Adversarial Networks (LSGAN), Deep Convolutional Generative Adversarial Networks (DCGAN), and Wasserstein Generative Adversarial Networks (WGAN). This paper also discusses, in particular, the quality of the images produced by various GANs with different parameters. For processing, we use a picture with a specific number and scale. The Structural Similarity Index (SSIM) and Mean Squared Error (MSE) will be used to measure image consistency. Between the generated image and the corresponding real image, the SSIM values will be compared. As a result, the images display a strong similarity to the real image when using more training images. LSGAN outperformed other GAN models in the experiment with maximum SSIM values achieved using 200 images as inputs, 2000 epochs, and size 32 × 32.

scholarly journals Cell Morphology-Guided De Novo Hit Design by Conditioning Generative Adversarial Networks on Phenotypic Image Features

2020 ◽  
Author(s):  
Oscar Méndez-Lucio ◽  
Paula Andrea Marin Zapata ◽  
Joerg Wichard ◽  
David Rouquié ◽  
Djork-Arné Clevert
Keyword(s):  
Small Molecules ◽  
De Novo ◽  
Molecular Targets ◽  
De Novo Design ◽  
Image Features ◽  
Mitigation Strategy ◽  
Generative Adversarial Networks ◽  
Target Information ◽  
Adversarial Networks ◽  
First Time

Developing new small molecules that are bioactive is time-consuming, costly and rarely successful. As a mitigation strategy, we apply, for the first time, generative adversarial networks to de novo design of small molecules using a phenotype-based drug discovery approach. We trained our model on a set of 30,000 compounds and their respective morphological profiles extracted from high content images; no target information was used to train the model. Using this approach, we were able to automatically design agonist-like compounds of different molecular targets.

scholarly journals Inverse Airfoil Design Method for Generating Varieties of Smooth Airfoils Using Conditional WGAN-GP

2021 ◽  
Author(s):  
Kazuo Yonekura ◽  
Nozomu Miyamoto ◽  
Katsuyuki Suzuki
Keyword(s):  
Machine Learning ◽  
Design Method ◽  
Lift Coefficient ◽  
Flow Analysis ◽  
Generative Adversarial Networks ◽  
Learning Models ◽  
Smoothing Methods ◽  
Adversarial Networks ◽  
Proposed Model ◽  
Machine Learning Models

Abstract Machine learning models are recently utilized for airfoil shape generation methods. It is desired to obtain airfoil shapes that satisfies required lift coefficient. Generative adversarial networks (GAN) output reasonable airfoil shapes. However, shapes obtained from ordinal GAN models are not smooth, and they need smoothing before flow analysis. Therefore, the models need to be coupled with B'ezier curves or other smoothing methods to obtain smooth shapes. Generating shapes without any smoothing methods is challenging. In this study, we employed conditional Wasserstein GAN with gradient penalty (CWGAN-GP) to generate airfoil shapes, and the obtained shapes are as smooth as those obtained using smoothing methods. With the proposed method, no additional smoothing method is needed to generate airfoils. Moreover, the proposed model outputs shapes that satisfy the lift coefficient requirements.

Sign in / Sign up

Export Citation Format

Share Document