scholarly journals A Foreground-Aware Framework for Local Face Attribute Transfer

Entropy ◽  
2021 ◽  
Vol 23 (5) ◽  
pp. 615
Author(s):  
Yuanbin Fu ◽  
Jiayi Ma ◽  
Xiaojie Guo
Keyword(s):  
Social Media ◽  
State Of The Art ◽  
Input Image ◽  
Experimental Results ◽  
Reference Image ◽  
Background Region ◽  
High Quality ◽  
Style Transfer ◽  
Internet Users ◽  
Foreground Region

In the context of social media, large amounts of headshot photos are taken everyday. Unfortunately, in addition to laborious editing and modification, creating a visually compelling photographic masterpiece for sharing requires advanced professional skills, which are difficult for ordinary Internet users. Though there are many algorithms automatically and globally transferring the style from one image to another, they fail to respect the semantics of the scene and are unable to allow users to merely transfer the attributes of one or two face organs in the foreground region leaving the background region unchanged. To overcome this problem, we developed a novel framework for semantically meaningful local face attribute transfer, which can flexibly transfer the local attribute of a face organ from the reference image to a semantically equivalent organ in the input image, while preserving the background. Our method involves warping the reference photo to match the shape, pose, location, and expression of the input image. The fusion of the warped reference image and input image is then taken as the initialized image for a neural style transfer algorithm. Our method achieves better performance in terms of inception score (3.81) and Fréchet inception distance (80.31), which is about 10% higher than those of competitors, indicating that our framework is capable of producing high-quality and photorealistic attribute transfer results. Both theoretical findings and experimental results are provided to demonstrate the efficacy of the proposed framework, reveal its superiority over other state-of-the-art alternatives.

scholarly journals Monolingual Transfer Learning via Bilingual Translators for Style-Sensitive Paraphrase Generation

2020 ◽  
Vol 34 (05) ◽  
pp. 8042-8049
Author(s):  
Tomoyuki Kajiwara ◽  
Biwa Miura ◽  
Yuki Arase
Keyword(s):  
Machine Translation ◽  
Transfer Learning ◽  
State Of The Art ◽  
Experimental Results ◽  
Fine Tuning ◽  
Training Methods ◽  
High Quality ◽  
Style Transfer ◽  
Parallel Corpus ◽  
Paraphrase Generation

We tackle the low-resource problem in style transfer by employing transfer learning that utilizes abundantly available raw corpora. Our method consists of two steps: pre-training learns to generate a semantically equivalent sentence with an input assured grammaticality, and fine-tuning learns to add a desired style. Pre-training has two options, auto-encoding and machine translation based methods. Pre-training based on AutoEncoder is a simple way to learn these from a raw corpus. If machine translators are available, the model can learn more diverse paraphrasing via roundtrip translation. After these, fine-tuning achieves high-quality paraphrase generation even in situations where only 1k sentence pairs of the parallel corpus for style transfer is available. Experimental results of formality style transfer indicated the effectiveness of both pre-training methods and the method based on roundtrip translation achieves state-of-the-art performance.

Language-Independent Type Inference of the Instances from Multilingual Wikipedia

2019 ◽  
Vol 15 (2) ◽  
pp. 22-46 ◽  
Author(s):  
Tianxing Wu ◽  
Guilin Qi ◽  
Bin Luo ◽  
Lei Zhang ◽  
Haofen Wang
Keyword(s):  
Knowledge Base ◽  
State Of The Art ◽  
The State ◽  
Type Inference ◽  
Experimental Results ◽  
High Quality ◽  
New Approach ◽  
Semantic Associations ◽  
Quality Type ◽  
Type Information

Extracting knowledge from Wikipedia has attracted much attention in recent ten years. One of the most valuable kinds of knowledge is type information, which refers to the axioms stating that an instance is of a certain type. Current approaches for inferring the types of instances from Wikipedia mainly rely on some language-specific rules. Since these rules cannot catch the semantic associations between instances and classes (i.e. candidate types), it may lead to mistakes and omissions in the process of type inference. The authors propose a new approach leveraging attributes to perform language-independent type inference of the instances from Wikipedia. The proposed approach is applied to the whole English and Chinese Wikipedia, which results in the first version of MulType (Multilingual Type Information), a knowledge base describing the types of instances from multilingual Wikipedia. Experimental results show that not only the proposed approach outperforms the state-of-the-art comparison methods, but also MulType contains lots of new and high-quality type information.

scholarly journals Consistent Video Style Transfer via Compound Regularization

2020 ◽  
Vol 34 (07) ◽  
pp. 12233-12240
Author(s):  
Wenjing Wang ◽  
Jizheng Xu ◽  
Li Zhang ◽  
Yue Wang ◽  
Jiaying Liu
Keyword(s):  
State Of The Art ◽  
Video Data ◽  
Experimental Results ◽  
Temporal Consistency ◽  
Challenging Problem ◽  
Style Transfer ◽  
Single Frame ◽  
Optical Flows ◽  
Training Strategies ◽  
Art Style

Recently, neural style transfer has drawn many attentions and significant progresses have been made, especially for image style transfer. However, flexible and consistent style transfer for videos remains a challenging problem. Existing training strategies, either using a significant amount of video data with optical flows or introducing single-frame regularizers, have limited performance on real videos. In this paper, we propose a novel interpretation of temporal consistency, based on which we analyze the drawbacks of existing training strategies; and then derive a new compound regularization. Experimental results show that the proposed regularization can better balance the spatial and temporal performance, which supports our modeling. Combining with the new cost formula, we design a zero-shot video style transfer framework. Moreover, for better feature migration, we introduce a new module to dynamically adjust inter-channel distributions. Quantitative and qualitative results demonstrate the superiority of our method over other state-of-the-art style transfer methods. Our project is publicly available at: https://daooshee.github.io/CompoundVST/.

scholarly journals Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning

2019 ◽  
Vol 33 ◽  
pp. 6722-6729 ◽  
Author(s):  
Ziming Li ◽  
Julia Kiseleva ◽  
Maarten De Rijke
Keyword(s):  
Reinforcement Learning ◽  
State Of The Art ◽  
The State ◽  
Experimental Results ◽  
Imitation Learning ◽  
Local Optimum ◽  
Inverse Reinforcement Learning ◽  
High Quality ◽  
Overall Performance

The performance of adversarial dialogue generation models relies on the quality of the reward signal produced by the discriminator. The reward signal from a poor discriminator can be very sparse and unstable, which may lead the generator to fall into a local optimum or to produce nonsense replies. To alleviate the first problem, we first extend a recently proposed adversarial dialogue generation method to an adversarial imitation learning solution. Then, in the framework of adversarial inverse reinforcement learning, we propose a new reward model for dialogue generation that can provide a more accurate and precise reward signal for generator training. We evaluate the performance of the resulting model with automatic metrics and human evaluations in two annotation settings. Our experimental results demonstrate that our model can generate more high-quality responses and achieve higher overall performance than the state-of-the-art.

scholarly journals MetaCoAG: Binning Metagenomic Contigs via Composition, Coverage and Assembly Graphs

2021 ◽  
Author(s):  
Vijini Mallawaarachchi ◽  
Yu Lin
Keyword(s):  
Microbial Communities ◽  
De Novo ◽  
State Of The Art ◽  
Genetic Material ◽  
Single Copy ◽  
Experimental Results ◽  
Marker Genes ◽  
High Quality ◽  
Second Best ◽  
Direct Use

ABSTRACTMetagenomics binning has allowed us to study and characterize various genetic material of different species and gain insights into microbial communities. While existing binning tools bin metagenomics de novo assemblies, they do not make use of the assembly graphs that produce such assemblies. Here we propose MetaCoAG, a tool that utilizes assembly graphs with the composition and coverage information to bin metagenomic contigs. MetaCoAG uses single-copy marker genes to estimate the number of initial bins, assigns contigs into bins iteratively and adjusts the number of bins dynamically throughout the binning process. Experimental results on simulated and real datasets demonstrate that MetaCoAG significantly outperforms state-of-the-art binning tools, producing more high-quality bins than the second-best tool, with an average median F1-score of 88.40%. To the best of our knowledge, MetaCoAG is the first stand-alone binning tool to make direct use of the assembly graph information. MetaCoAG is available at https://github.com/Vini2/MetaCoAG.

scholarly journals Direction-aware Feature-level Frequency Decomposition for Single Image Deraining

2021 ◽  
Author(s):  
Sen Deng ◽  
Yidan Feng ◽  
Mingqiang Wei ◽  
Haoran Xie ◽  
Yiping Chen ◽  
...  
Keyword(s):  
High Frequency ◽  
State Of The Art ◽  
Low Frequency ◽  
Communication Channels ◽  
Input Image ◽  
Experimental Results ◽  
Training Procedure ◽  
Single Image ◽  
Frequency Decomposition

We present a novel direction-aware feature-level frequency decomposition network for single image deraining. Compared with existing solutions, the proposed network has three compelling characteristics. First, unlike previous algorithms, we propose to perform frequency decomposition at feature-level instead of image-level, allowing both low-frequency maps containing structures and high-frequency maps containing details to be continuously refined during the training procedure. Second, we further establish communication channels between low-frequency maps and high-frequency maps to interactively capture structures from high-frequency maps and add them back to low-frequency maps and, simultaneously, extract details from low-frequency maps and send them back to high-frequency maps, thereby removing rain streaks while preserving more delicate features in the input image. Third, different from existing algorithms using convolutional filters consistent in all directions, we propose a direction-aware filter to capture the direction of rain streaks in order to more effectively and thoroughly purge the input images of rain streaks. We extensively evaluate the proposed approach in three representative datasets and experimental results corroborate our approach consistently outperforms state-of-the-art deraining algorithms.

scholarly journals TET-GAN: Text Effects Transfer via Stylization and Destylization

2019 ◽  
Vol 33 ◽  
pp. 1238-1245 ◽  
Author(s):  
Shuai Yang ◽  
Jiaying Liu ◽  
Wenjing Wang ◽  
Zongming Guo
Keyword(s):  
Network Design ◽  
State Of The Art ◽  
The State ◽  
High Quality ◽  
Style Transfer ◽  
Feature Representations ◽  
Art Methods ◽  
Matching Criteria

Text effects transfer technology automatically makes the text dramatically more impressive. However, previous style transfer methods either study the model for general style, which cannot handle the highly-structured text effects along the glyph, or require manual design of subtle matching criteria for text effects. In this paper, we focus on the use of the powerful representation abilities of deep neural features for text effects transfer. For this purpose, we propose a novel Texture Effects Transfer GAN (TET-GAN), which consists of a stylization subnetwork and a destylization subnetwork. The key idea is to train our network to accomplish both the objective of style transfer and style removal, so that it can learn to disentangle and recombine the content and style features of text effects images. To support the training of our network, we propose a new text effects dataset with as much as 64 professionally designed styles on 837 characters. We show that the disentangled feature representations enable us to transfer or remove all these styles on arbitrary glyphs using one network. Furthermore, the flexible network design empowers TET-GAN to efficiently extend to a new text style via oneshot learning where only one example is required. We demonstrate the superiority of the proposed method in generating high-quality stylized text over the state-of-the-art methods.

Language-Independent Type Inference of the Instances from Multilingual Wikipedia

2022 ◽  
pp. 580-606
Author(s):  
Tianxing Wu ◽  
Guilin Qi ◽  
Bin Luo ◽  
Lei Zhang ◽  
Haofen Wang
Keyword(s):  
Knowledge Base ◽  
State Of The Art ◽  
The State ◽  
Type Inference ◽  
Experimental Results ◽  
High Quality ◽  
New Approach ◽  
Semantic Associations ◽  
Quality Type ◽  
Type Information

Extracting knowledge from Wikipedia has attracted much attention in recent ten years. One of the most valuable kinds of knowledge is type information, which refers to the axioms stating that an instance is of a certain type. Current approaches for inferring the types of instances from Wikipedia mainly rely on some language-specific rules. Since these rules cannot catch the semantic associations between instances and classes (i.e. candidate types), it may lead to mistakes and omissions in the process of type inference. The authors propose a new approach leveraging attributes to perform language-independent type inference of the instances from Wikipedia. The proposed approach is applied to the whole English and Chinese Wikipedia, which results in the first version of MulType (Multilingual Type Information), a knowledge base describing the types of instances from multilingual Wikipedia. Experimental results show that not only the proposed approach outperforms the state-of-the-art comparison methods, but also MulType contains lots of new and high-quality type information.

Fast Depth Generating Algorithm From Binocular Images

2013 ◽  
Vol 284-287 ◽  
pp. 3102-3105
Author(s):  
Fang Hsuan Cheng ◽  
Tzu Hao Kuo
Keyword(s):  
Edge Detection ◽  
Line Segment ◽  
Stereo Matching ◽  
Color Difference ◽  
Experimental Results ◽  
Reference Image ◽  
Depth Information ◽  
High Quality ◽  
Line Segments

This paper proposes a line segment method to estimate the depth information from a pair of rectified images. The proposed method can achieve fast and high quality stereo-matching. This method first uses a simple edge detection to find out the line segments in the reference image and then calculates the color difference of each line segment from binocular images. The last step is to find out the minimum difference of each line segment as the corresponding points. From the experimental results, it is proved that the proposed method can fast and accurately generate the depth information from binocular images.

Image-Based Simulation for Snowy Scenes

2011 ◽  
Vol 58-60 ◽  
pp. 1329-1335
Author(s):  
Hai Ming Yin ◽  
Yuan Wang Wei ◽  
Yong Gang Li
Keyword(s):  
Computational Complexity ◽  
Image Data ◽  
Experimental Results ◽  
Reference Image ◽  
Color Information ◽  
Target Image ◽  
Color Transfer ◽  
High Quality ◽  
Snow Model

In this paper, we propose an image-based approach for the simulation of outdoor snowy scenes. Using a snowy image as the reference, we extract the snow covered regions from the reference through a snow model and achieve the snow color information. Then the target image will get the snow color information and take on snowy appearance through a color transfer procedure. To solve the problem of different data distributions between the reference and the target in the procedure, power transform and modulus transform are employed to adjust the image data according to the snow information derived from the reference image data. The experimental results indicate this approach can simulate the snowy scenes with high quality while greatly reducing the computational complexity of simulation compared with the traditional algorithms.

Sign in / Sign up

Export Citation Format

Share Document