Feature Level Fusion for Bimodal Facial Action Unit Recognition

2015 ◽  
Author(s):  
Zibo Meng ◽  
Shizhong Han ◽  
Min Chen ◽  
Yan Tong
Keyword(s):  
Action Unit ◽  
Facial Action ◽  
Feature Level Fusion ◽  
Level Fusion

Audiovisual Facial Action Unit Recognition using Feature Level Fusion

2016 ◽  
Vol 7 (1) ◽  
pp. 60-76 ◽  
Author(s):  
Zibo Meng ◽  
Shizhong Han ◽  
Min Chen ◽  
Yan Tong
Keyword(s):  
Time Shift ◽  
Visual Features ◽  
Visual Channel ◽  
Facial Action ◽  
Audio Channel ◽  
Feature Level Fusion ◽  
Fusion Methods ◽  
The Difference ◽  
Facial Images ◽  
Level Fusion

Recognizing facial actions is challenging, especially when they are accompanied with speech. Instead of employing information solely from the visual channel, this work aims to exploit information from both visual and audio channels in recognizing speech-related facial action units (AUs). In this work, two feature-level fusion methods are proposed. The first method is based on a kind of human-crafted visual feature. The other method utilizes visual features learned by a deep convolutional neural network (CNN). For both methods, features are independently extracted from visual and audio channels and aligned to handle the difference in time scales and the time shift between the two signals. These temporally aligned features are integrated via feature-level fusion for AU recognition. Experimental results on a new audiovisual AU-coded dataset have demonstrated that both fusion methods outperform their visual counterparts in recognizing speech-related AUs. The improvement is more impressive with occlusions on the facial images, which would not affect the audio channel.

Audiovisual Facial Action Unit Recognition Using Feature Level Fusion

2018 ◽  
pp. 636-655
Author(s):  
Zibo Meng ◽  
Shizhong Han ◽  
Min Chen ◽  
Yan Tong
Keyword(s):  
Time Shift ◽  
Visual Feature ◽  
Visual Channel ◽  
Facial Action ◽  
Audio Channel ◽  
Feature Level Fusion ◽  
Fusion Methods ◽  
The Difference ◽  
Facial Images ◽  
Level Fusion

Recognizing facial actions is challenging, especially when they are accompanied with speech. Instead of employing information solely from the visual channel, this work aims to exploit information from both visual and audio channels in recognizing speech-related facial action units (AUs). In this work, two feature-level fusion methods are proposed. The first method is based on a kind of human-crafted visual feature. The other method utilizes visual features learned by a deep convolutional neural network (CNN). For both methods, features are independently extracted from visual and audio channels and aligned to handle the difference in time scales and the time shift between the two signals. These temporally aligned features are integrated via feature-level fusion for AU recognition. Experimental results on a new audiovisual AU-coded dataset have demonstrated that both fusion methods outperform their visual counterparts in recognizing speech-related AUs. The improvement is more impressive with occlusions on the facial images, which would not affect the audio channel.

Tunneled Latent Variables Method for Facial Action Unit Tracking

2009 ◽  
Vol 35 (2) ◽  
pp. 198-201 ◽  
Author(s):  
Lei WANG ◽  
Bei-Ji ZOU ◽  
Xiao-Ning PENG
Keyword(s):  
Latent Variables ◽  
Action Unit ◽  
Facial Action

The Review of Feature Level Fusion of Multi-Focused Images Using Wavelets

2010 ◽  
Vol 2 (1) ◽  
pp. 28-38 ◽  
Author(s):  
K. Kannan ◽  
S. Arumuga Perumal ◽  
K. Arulmozhi
Keyword(s):  
Feature Level Fusion ◽  
Level Fusion

Multi-view facial action unit detection via DenseNets and CapsNets

2021 ◽  
Author(s):  
Dakai Ren ◽  
Xiangmin Wen ◽  
Jiazhong Chen ◽  
Yu Han ◽  
Shiqi Zhang
Keyword(s):  
Action Unit ◽  
Facial Action ◽  
Action Unit Detection

scholarly journals Assessing Automated Facial Action Unit Detection Systems for Analyzing Cross-Domain Facial Expression Databases

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 4222
Author(s):  
Shushi Namba ◽  
Wataru Sato ◽  
Masaki Osumi ◽  
Koh Shimokawa
Keyword(s):  
Affective Computing ◽  
Detection System ◽  
Characteristic Curve ◽  
Unmet Need ◽  
Systematic Evaluation ◽  
Dynamic Mode ◽  
Coding System ◽  
Static Mode ◽  
Action Unit ◽  
Facial Action

In the field of affective computing, achieving accurate automatic detection of facial movements is an important issue, and great progress has already been made. However, a systematic evaluation of systems that now have access to the dynamic facial database remains an unmet need. This study compared the performance of three systems (FaceReader, OpenFace, AFARtoolbox) that detect each facial movement corresponding to an action unit (AU) derived from the Facial Action Coding System. All machines could detect the presence of AUs from the dynamic facial database at a level above chance. Moreover, OpenFace and AFAR provided higher area under the receiver operating characteristic curve values compared to FaceReader. In addition, several confusion biases of facial components (e.g., AU12 and AU14) were observed to be related to each automated AU detection system and the static mode was superior to dynamic mode for analyzing the posed facial database. These findings demonstrate the features of prediction patterns for each system and provide guidance for research on facial expressions.

Sign in / Sign up

Export Citation Format

Share Document