Realistic head motion synthesis for an image-based talking head

2011 ◽  
Author(s):  
Kang Liu ◽  
Joern Ostermann
Keyword(s):  
Head Motion ◽  
Motion Synthesis ◽  
Talking Head

Head motion synthesis from speech using deep neural networks

2014 ◽  
Vol 74 (22) ◽  
pp. 9871-9888 ◽  
Author(s):  
Chuang Ding ◽  
Lei Xie ◽  
Pengcheng Zhu
Keyword(s):  
Neural Networks ◽  
Deep Neural Networks ◽  
Head Motion ◽  
Motion Synthesis

scholarly journals Speech-driven head motion synthesis using neural networks

2014 ◽  
Author(s):  
Chuang Ding ◽  
Pengcheng Zhu ◽  
Lei Xie ◽  
Dongmei Jiang ◽  
Zhong-Hua Fu
Keyword(s):  
Neural Networks ◽  
Head Motion ◽  
Motion Synthesis

Template-warping based speech driven head motion synthesis

2013 ◽  
Author(s):  
David Adam Braude ◽  
Hiroshi Shimodaira ◽  
Atef Ben-Youssef
Keyword(s):  
Head Motion ◽  
Motion Synthesis

scholarly journals Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion

2021 ◽  
Author(s):  
Suzhen Wang ◽  
Lincheng Li ◽  
Yu Ding ◽  
Changjie Fan ◽  
Xin Yu
Keyword(s):  
Low Frequency ◽  
Head Movements ◽  
Reference Image ◽  
Head Motion ◽  
Motion Field ◽  
Temporal Consistency ◽  
Field Representation ◽  
Speech Prosody ◽  
Talking Head ◽  
Generation Network

We propose an audio-driven talking-head method to generate photo-realistic talking-head videos from a single reference image. In this work, we tackle two key challenges: (i) producing natural head motions that match speech prosody, and (ii)} maintaining the appearance of a speaker in a large head motion while stabilizing the non-face regions. We first design a head pose predictor by modeling rigid 6D head movements with a motion-aware recurrent neural network (RNN). In this way, the predicted head poses act as the low-frequency holistic movements of a talking head, thus allowing our latter network to focus on detailed facial movement generation. To depict the entire image motions arising from audio, we exploit a keypoint based dense motion field representation. Then, we develop a motion field generator to produce the dense motion fields from input audio, head poses, and a reference image. As this keypoint based representation models the motions of facial regions, head, and backgrounds integrally, our method can better constrain the spatial and temporal consistency of the generated videos. Finally, an image generation network is employed to render photo-realistic talking-head videos from the estimated keypoint based motion fields and the input reference image. Extensive experiments demonstrate that our method produces videos with plausible head motions, synchronized facial expressions, and stable backgrounds and outperforms the state-of-the-art.

Natural head motion synthesis driven by acoustic prosodic features

2005 ◽  
Vol 16 (3-4) ◽  
pp. 283-290 ◽  
Author(s):  
Carlos Busso ◽  
Zhigang Deng ◽  
Ulrich Neumann ◽  
Shrikanth Narayanan
Keyword(s):  
Head Motion ◽  
Prosodic Features ◽  
Motion Synthesis

scholarly journals Articulatory features for speech-driven head motion synthesis

2013 ◽  
Author(s):  
Atef Ben-Youssef ◽  
Hiroshi Shimodaira ◽  
David Adam Braude
Keyword(s):  
Head Motion ◽  
Motion Synthesis ◽  
Articulatory Features
Sign in / Sign up

Export Citation Format

Share Document