scholarly journals Cada-Fvae-Gan: Adversarial Training for Few-Shot Event Detection

2020 ◽  
Author(s):  
Xiaoxiang Zhu ◽  
Mengshu Hou ◽  
Xiaoyang Zeng ◽  
Hao Zhu
Keyword(s):  
Event Detection ◽  
Semantic Features ◽  
Learning Framework ◽  
Latent Space ◽  
Variational Autoencoder ◽  
Adversarial Training ◽  
Human Effort ◽  
Latent Representations ◽  
Typical Solution

Most supervised systems of event detection (ED) task reply heavily on manual annotations and suffer from high-cost human effort when applied to new event types. To tackle this general problem, we turn our attention to few-shot learning (FSL). As a typical solution to FSL, cross-modal feature generation based frameworks achieve promising performance on images classification, which inspires us to advance this approach to ED task. In this work, we propose a model which extracts latent semantic features from event mentions, type structures and type names, then these three modalities are mapped into a shared low-dimension latent space by modality-specific aligned variational autoencoder enhanced by adversarial training. We evaluate the quality of our latent representations by training a CNN classifier to perform ED task. Experiments conducted on ACE2005 dataset show an improvement with 12.67% on F1-score when introducing adversarial training to VAE model, and our method is comparable with existing transfer learning framework for ED.

Penalized Variational Autoencoder for Molecular Design

2019 ◽  
Author(s):  
Sadegh Mohammadi ◽  
Bing O'Dowd ◽  
Christian Paulitz-Erdmann ◽  
Linus Goerlitz
Keyword(s):  
Molecular Design ◽  
Biological Properties ◽  
Biological Information ◽  
Penalty Term ◽  
Cross Domain ◽  
Latent Space ◽  
Variational Autoencoder ◽  
Linear String ◽  
Weight Penalty

Variational autoencoders have emerged as one of the most common approaches for automating molecular generation. We seek to learn a cross-domain latent space capturing chemical and biological information, simultaneously. To do so, we introduce the Penalized Variational Autoencoder which directly operates on SMILES, a linear string representation of molecules, with a weight penalty term in the decoder to address the imbalance in the character distribution of SMILES strings. We find that this greatly improves upon previous variational autoencoder approaches in the quality of the latent space and the generalization ability of the latent space to new chemistry. Next, we organize the latent space according to chemical and biological properties by jointly training the Penalized Variational Autoencoder with linear units. Extensive experiments on a range of tasks, including reconstruction, validity, and transferability demonstrates that the proposed methods here substantially outperform previous SMILES and graph-based methods, as well as introduces a new way to generate molecules from a set of desired properties, without prior knowledge of a chemical structure.

scholarly journals Structured variational inference for simulating populations of radio galaxies

2021 ◽  
Vol 503 (3) ◽  
pp. 3351-3370
Author(s):  
David J Bastien ◽  
Anna M M Scaife ◽  
Hongming Tang ◽  
Micah Bowles ◽  
Fiona Porter
Keyword(s):  
Data Augmentation ◽  
Radio Galaxies ◽  
Variational Inference ◽  
Postage Stamp ◽  
Synthetic Populations ◽  
Latent Space ◽  
Variational Autoencoder ◽  
Decoder Architecture ◽  
Fully Connected

ABSTRACT We present a model for generating postage stamp images of synthetic Fanaroff–Riley Class I and Class II radio galaxies suitable for use in simulations of future radio surveys such as those being developed for the Square Kilometre Array. This model uses a fully connected neural network to implement structured variational inference through a variational autoencoder and decoder architecture. In order to optimize the dimensionality of the latent space for the autoencoder, we introduce the radio morphology inception score (RAMIS), a quantitative method for assessing the quality of generated images, and discuss in detail how data pre-processing choices can affect the value of this measure. We examine the 2D latent space of the VAEs and discuss how this can be used to control the generation of synthetic populations, whilst also cautioning how it may lead to biases when used for data augmentation.

scholarly journals Regularizing Variational Autoencoder with Diversity and Uncertainty Awareness

2021 ◽  
Author(s):  
Dazhong Shen ◽  
Chuan Qin ◽  
Chao Wang ◽  
Hengshu Zhu ◽  
Enhong Chen ◽  
...  
Keyword(s):  
Latent Variables ◽  
State Of The Art ◽  
Likelihood Estimation ◽  
Generative Models ◽  
Latent Space ◽  
Variational Autoencoder ◽  
Benchmark Datasets ◽  
Classification Tasks ◽  
Latent Representations ◽  
Low Uncertainty

As one of the most popular generative models, Variational Autoencoder (VAE) approximates the posterior of latent variables based on amortized variational inference. However, when the decoder network is sufficiently expressive, VAE may lead to posterior collapse; that is, uninformative latent representations may be learned. To this end, in this paper, we propose an alternative model, DU-VAE, for learning a more Diverse and less Uncertain latent space, and thus the representation can be learned in a meaningful and compact manner. Specifically, we first theoretically demonstrate that it will result in better latent space with high diversity and low uncertainty awareness by controlling the distribution of posterior’s parameters across the whole data accordingly. Then, without the introduction of new loss terms or modifying training strategies, we propose to exploit Dropout on the variances and Batch-Normalization on the means simultaneously to regularize their distributions implicitly. Furthermore, to evaluate the generalization effect, we also exploit DU-VAE for inverse autoregressive flow based-VAE (VAE-IAF) empirically. Finally, extensive experiments on three benchmark datasets clearly show that our approach can outperform state-of-the-art baselines on both likelihood estimation and underlying classification tasks.

scholarly journals MixPoet: Diverse Poetry Generation via Learning Controllable Mixed Latent Space

2020 ◽  
Vol 34 (05) ◽  
pp. 9450-9457
Author(s):  
Xiaoyuan Yi ◽  
Ruoyu Li ◽  
Cheng Yang ◽  
Wenhao Li ◽  
Maosong Sun
Keyword(s):  
Latent Variable ◽  
Life Experience ◽  
Influence Factor ◽  
Historical Background ◽  
Chinese Poetry ◽  
Neural Models ◽  
Latent Space ◽  
Variational Autoencoder ◽  
Adversarial Training ◽  
Novel Model

As an essential step towards computer creativity, automatic poetry generation has gained increasing attention these years. Though recent neural models make prominent progress in some criteria of poetry quality, generated poems still suffer from the problem of poor diversity. Related literature researches show that different factors, such as life experience, historical background, etc., would influence composition styles of poets, which considerably contributes to the high diversity of human-authored poetry. Inspired by this, we propose MixPoet, a novel model that absorbs multiple factors to create various styles and promote diversity. Based on a semi-supervised variational autoencoder, our model disentangles the latent space into some subspaces, with each conditioned on one influence factor by adversarial training. In this way, the model learns a controllable latent variable to capture and mix generalized factor-related properties. Different factor mixtures lead to diverse styles and hence further differentiate generated poems from each other. Experiment results on Chinese poetry demonstrate that MixPoet improves both diversity and quality against three state-of-the-art models.

scholarly journals OptAGAN: Entropy-based Finetuning on Text VAE-GAN

2021 ◽  
Author(s):  
Paolo Tirotta ◽  
Stefano Lodi
Keyword(s):  
Language Processing ◽  
Generative Adversarial Networks ◽  
Word Generation ◽  
Adversarial Networks ◽  
Current State ◽  
Latent Space ◽  
Variational Autoencoder ◽  
Maximum Likelihood Methods ◽  
High Level

Transfer learning through large pre-trained models has changed the landscape of current applications in natural language processing (NLP). Recently Optimus, a variational autoencoder (VAE) which combines two pre-trained models, BERT and GPT-2, has been released, and its combination with generative adversarial networks (GANs) has been shown to produce novel, yet very human-looking text. The Optimus and GANs combination avoids the troublesome application of GANs to the discrete domain of text, and prevents the exposure bias of standard maximum likelihood methods. We combine the training of GANs in the latent space, with the finetuning of the decoder of Optimus for single word generation. This approach lets us model both the high-level features of the sentences, and the low-level word-by-word generation. We finetune using reinforcement learning (RL) by exploiting the structure of GPT-2 and by adding entropy-based intrinsically motivated rewards to balance between quality and diversity. We benchmark the results of the VAE-GAN model, and show the improvements brought by our RL finetuning on three widely used datasets for text generation, with results that greatly surpass the current state-of-the-art for the quality of the generated texts.

Penalized Variational Autoencoder for Molecular Design

2019 ◽  
Author(s):  
Sadegh Mohammadi ◽  
Bing O'Dowd ◽  
Christian Paulitz-Erdmann ◽  
Linus Goerlitz
Keyword(s):  
Molecular Design ◽  
Biological Properties ◽  
Biological Information ◽  
Penalty Term ◽  
Cross Domain ◽  
Latent Space ◽  
Variational Autoencoder ◽  
Linear String ◽  
Weight Penalty

Variational autoencoders have emerged as one of the most common approaches for automating molecular generation. We seek to learn a cross-domain latent space capturing chemical and biological information, simultaneously. To do so, we introduce the Penalized Variational Autoencoder which directly operates on SMILES, a linear string representation of molecules, with a weight penalty term in the decoder to address the imbalance in the character distribution of SMILES strings. We find that this greatly improves upon previous variational autoencoder approaches in the quality of the latent space and the generalization ability of the latent space to new chemistry. Next, we organize the latent space according to chemical and biological properties by jointly training the Penalized Variational Autoencoder with linear units. Extensive experiments on a range of tasks, including reconstruction, validity, and transferability demonstrates that the proposed methods here substantially outperform previous SMILES and graph-based methods, as well as introduces a new way to generate molecules from a set of desired properties, without prior knowledge of a chemical structure.

Penalized Variational Autoencoder for Molecular Design

2019 ◽  
Author(s):  
Sadegh Mohammadi ◽  
Bing O'Dowd ◽  
Christian Paulitz-Erdmann ◽  
Linus Goerlitz
Keyword(s):  
Molecular Design ◽  
Biological Properties ◽  
Biological Information ◽  
Penalty Term ◽  
Cross Domain ◽  
Latent Space ◽  
Variational Autoencoder ◽  
Linear String ◽  
Weight Penalty

Variational autoencoders have emerged as one of the most common approaches for automating molecular generation. We seek to learn a cross-domain latent space capturing chemical and biological information, simultaneously. To do so, we introduce the Penalized Variational Autoencoder which directly operates on SMILES, a linear string representation of molecules, with a weight penalty term in the decoder to address the imbalance in the character distribution of SMILES strings. We find that this greatly improves upon previous variational autoencoder approaches in the quality of the latent space and the generalization ability of the latent space to new chemistry. Next, we organize the latent space according to chemical and biological properties by jointly training the Penalized Variational Autoencoder with linear units. Extensive experiments on a range of tasks, including reconstruction, validity, and transferability demonstrates that the proposed methods here substantially outperform previous SMILES and graph-based methods, as well as introduces a new way to generate molecules from a set of desired properties, without prior knowledge of a chemical structure.

scholarly journals Bayesian Learning of Latent Representations of Language Structures

2019 ◽  
Vol 45 (2) ◽  
pp. 199-228
Author(s):  
Yugo Murawaki
Keyword(s):  
Latent Variables ◽  
Bayesian Learning ◽  
Missing Values ◽  
Representation Learning ◽  
Model Parameters ◽  
Challenging Problem ◽  
Learning Framework ◽  
Proposed Model ◽  
Latent Space ◽  
Latent Representations

We borrow the concept of representation learning from deep learning research, and we argue that the quest for Greenbergian implicational universals can be reformulated as the learning of good latent representations of languages, or sequences of surface typological features. By projecting languages into latent representations and performing inference in the latent space, we can handle complex dependencies among features in an implicit manner. The most challenging problem in turning the idea into a concrete computational model is the alarmingly large number of missing values in existing typological databases. To address this problem, we keep the number of model parameters relatively small to avoid overfitting, adopt the Bayesian learning framework for its robustness, and exploit phylogenetically and/or spatially related languages as additional clues. Experiments show that the proposed model recovers missing values more accurately than others and that some latent variables exhibit phylogenetic and spatial signals comparable to those of surface features.

scholarly journals Deep Adversarial Multi-view Clustering Network

2019 ◽  
Author(s):  
Zhaoyang Li ◽  
Qianqian Wang ◽  
Zhiqiang Tao ◽  
Quanxue Gao ◽  
Zhaohua Yang
Keyword(s):  
Clustering Algorithms ◽  
Cluster Structure ◽  
Multiple Views ◽  
Intrinsic Structure ◽  
Common Structure ◽  
Latent Space ◽  
Adversarial Training ◽  
The Common ◽  
Latent Representations ◽  
Real World Datasets

Multi-view clustering has attracted increasing attention in recent years by exploiting common clustering structure across multiple views. Most existing multi-view clustering algorithms use shallow and linear embedding functions to learn the common structure of multi-view data. However, these methods cannot fully utilize the non-linear property of multi-view data, which is important to reveal complex cluster structure underlying multi-view data. In this paper, we propose a novel multi-view clustering method, named Deep Adversarial Multi-view Clustering (DAMC) network, to learn the intrinsic structure embedded in multi-view data. Specifically, our model adopts deep auto-encoders to learn latent representations shared by multiple views, and meanwhile leverages adversarial training to further capture the data distribution and disentangle the latent space. Experimental results on several real-world datasets demonstrate that the proposed method outperforms the state-of art methods.

scholarly journals Nodule Detection with Convolutional Neural Network Using Apache Spark and GPU Frameworks

2021 ◽  
Vol 11 (6) ◽  
pp. 2838
Author(s):  
Nikitha Johnsirani Venkatesan ◽  
Dong Ryeol Shin ◽  
Choon Sung Nam
Keyword(s):  
Neural Network ◽  
Radiation Dose ◽  
Convolutional Neural Network ◽  
Model Performance ◽  
Performance Comparison ◽  
Apache Spark ◽  
Training Time ◽  
Learning Framework ◽  
Proposed Model

In the pharmaceutical field, early detection of lung nodules is indispensable for increasing patient survival. We can enhance the quality of the medical images by intensifying the radiation dose. High radiation dose provokes cancer, which forces experts to use limited radiation. Using abrupt radiation generates noise in CT scans. We propose an optimal Convolutional Neural Network model in which Gaussian noise is removed for better classification and increased training accuracy. Experimental demonstration on the LUNA16 dataset of size 160 GB shows that our proposed method exhibit superior results. Classification accuracy, specificity, sensitivity, Precision, Recall, F1 measurement, and area under the ROC curve (AUC) of the model performance are taken as evaluation metrics. We conducted a performance comparison of our proposed model on numerous platforms, like Apache Spark, GPU, and CPU, to depreciate the training time without compromising the accuracy percentage. Our results show that Apache Spark, integrated with a deep learning framework, is suitable for parallel training computation with high accuracy.

Sign in / Sign up

Export Citation Format

Share Document