scholarly journals Visual Scene-Aware Hybrid and Multi-Modal Feature Aggregation for Facial Expression Recognition

Sensors ◽  
2020 ◽  
Vol 20 (18) ◽  
pp. 5184
Author(s):  
Min Kyu Lee ◽  
Dae Ha Kim ◽  
Byung Cheol Song
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Facial Expression Recognition ◽  
State Of The Art ◽  
Rapid Development ◽  
Expression Recognition ◽  
Fusion Method ◽  
Facial Landmark ◽  
Latent Features ◽  
Inter Frame

Facial expression recognition (FER) technology has made considerable progress with the rapid development of deep learning. However, conventional FER techniques are mainly designed and trained for videos that are artificially acquired in a limited environment, so they may not operate robustly on videos acquired in a wild environment suffering from varying illuminations and head poses. In order to solve this problem and improve the ultimate performance of FER, this paper proposes a new architecture that extends a state-of-the-art FER scheme and a multi-modal neural network that can effectively fuse image and landmark information. To this end, we propose three methods. To maximize the performance of the recurrent neural network (RNN) in the previous scheme, we first propose a frame substitution module that replaces the latent features of less important frames with those of important frames based on inter-frame correlation. Second, we propose a method for extracting facial landmark features based on the correlation between frames. Third, we propose a new multi-modal fusion method that effectively fuses video and facial landmark information at the feature level. By applying attention based on the characteristics of each modality to the features of the modality, novel fusion is achieved. Experimental results show that the proposed method provides remarkable performance, with 51.4% accuracy for the wild AFEW dataset, 98.5% accuracy for the CK+ dataset and 81.9% accuracy for the MMI dataset, outperforming the state-of-the-art networks.

scholarly journals The FaceChannel: A Fast and Furious Deep Neural Network for Facial Expression Recognition

2020 ◽  
Vol 1 (6) ◽  
Author(s):  
Pablo Barros ◽  
Nikhil Churamani ◽  
Alessandra Sciutti
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Facial Expression ◽  
Facial Expression Recognition ◽  
Deep Neural Networks ◽  
State Of The Art ◽  
Facial Features ◽  
Expression Recognition ◽  
Current State ◽  
Benchmark Datasets

AbstractCurrent state-of-the-art models for automatic facial expression recognition (FER) are based on very deep neural networks that are effective but rather expensive to train. Given the dynamic conditions of FER, this characteristic hinders such models of been used as a general affect recognition. In this paper, we address this problem by formalizing the FaceChannel, a light-weight neural network that has much fewer parameters than common deep neural networks. We introduce an inhibitory layer that helps to shape the learning of facial features in the last layer of the network and, thus, improving performance while reducing the number of trainable parameters. To evaluate our model, we perform a series of experiments on different benchmark datasets and demonstrate how the FaceChannel achieves a comparable, if not better, performance to the current state-of-the-art in FER. Our experiments include cross-dataset analysis, to estimate how our model behaves on different affective recognition conditions. We conclude our paper with an analysis of how FaceChannel learns and adapts the learned facial features towards the different datasets.

scholarly journals Three-Stream Convolutional Neural Network with Squeeze-and-Excitation Block for Near-Infrared Facial Expression Recognition

Electronics ◽  
2019 ◽  
Vol 8 (4) ◽  
pp. 385 ◽  
Author(s):  
Ying Chen ◽  
Zhihao Zhang ◽  
Lei Zhong ◽  
Tong Chen ◽  
Juxiang Chen ◽  
...  
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Facial Expression Recognition ◽  
Near Infrared ◽  
Recognition Accuracy ◽  
State Of The Art ◽  
Recognition Rate ◽  
Three Dimensional ◽  
Expression Recognition ◽  
Facial Expression Database

Near-infrared (NIR) facial expression recognition is resistant to illumination change. In this paper, we propose a three-stream three-dimensional convolution neural network with a squeeze-and-excitation (SE) block for NIR facial expression recognition. We fed each stream with different local regions, namely the eyes, nose, and mouth. By using an SE block, the network automatically allocated weights to different local features to further improve recognition accuracy. The experimental results on the Oulu-CASIA NIR facial expression database showed that the proposed method has a higher recognition rate than some state-of-the-art algorithms.

scholarly journals Iranian kinect face database (IKFDB): a color-depth based face database collected by kinect v.2 sensor

2021 ◽  
Vol 3 (1) ◽  
Author(s):  
Seyed Muhammad Hossein Mousavi ◽  
S. Younes Mirinezhad
Keyword(s):  
Neural Network ◽  
Facial Expression ◽  
Facial Expression Recognition ◽  
Depth Image ◽  
Sensor Technology ◽  
Support Vector ◽  
Expression Recognition ◽  
Face Database ◽  
Depth Data ◽  
Color Depth

AbstractThis study presents a new color-depth based face database gathered from different genders and age ranges from Iranian subjects. Using suitable databases, it is possible to validate and assess available methods in different research fields. This database has application in different fields such as face recognition, age estimation and Facial Expression Recognition and Facial Micro Expressions Recognition. Image databases based on their size and resolution are mostly large. Color images usually consist of three channels namely Red, Green and Blue. But in the last decade, another aspect of image type has emerged, named “depth image”. Depth images are used in calculating range and distance between objects and the sensor. Depending on the depth sensor technology, it is possible to acquire range data differently. Kinect sensor version 2 is capable of acquiring color and depth data simultaneously. Facial expression recognition is an important field in image processing, which has multiple uses from animation to psychology. Currently, there is a few numbers of color-depth (RGB-D) facial micro expressions recognition databases existing. With adding depth data to color data, the accuracy of final recognition will be increased. Due to the shortage of color-depth based facial expression databases and some weakness in available ones, a new and almost perfect RGB-D face database is presented in this paper, covering Middle-Eastern face type. In the validation section, the database will be compared with some famous benchmark face databases. For evaluation, Histogram Oriented Gradients features are extracted, and classification algorithms such as Support Vector Machine, Multi-Layer Neural Network and a deep learning method, called Convolutional Neural Network or are employed. The results are so promising.

Conditional convolution neural network enhanced random forest for facial expression recognition

2018 ◽  
Vol 84 ◽  
pp. 251-261 ◽  
Author(s):  
Yuanyuan Liu ◽  
Xiaohui Yuan ◽  
Xi Gong ◽  
Zhong Xie ◽  
Fang Fang ◽  
...  
Keyword(s):  
Neural Network ◽  
Random Forest ◽  
Facial Expression ◽  
Facial Expression Recognition ◽  
Convolution Neural Network ◽  
Expression Recognition

scholarly journals PROSES EKSTRAKSI DAN KLASIFIKASI CITRA EMOSI MENGGUNAKAN METODE PCA DAN CNN

JOUTICA ◽  
2021 ◽  
Vol 6 (2) ◽  
pp. 484
Author(s):  
Resty Wulanningrum ◽  
Anggi Nur Fadzila ◽  
Danar Putra Pamungkas
Keyword(s):  
Neural Network ◽  
Principal Component Analysis ◽  
Facial Expression ◽  
Convolutional Neural Network ◽  
Facial Expression Recognition ◽  
Principal Component ◽  
Component Analysis ◽  
Expression Recognition

Manusia secara alami menggunakan ekspresi wajah untuk berkomunikasi dan menunjukan emosi mereka dalam berinteraksi sosial. Ekspresi wajah termasuk kedalam komunikasi non-verbal yang dapat menyampaikan keadaan emosi seseorang kepada orang yang telah mengamatinya. Penelitian ini menggunakan metode Principal Component Analysis (PCA) untuk proses ekstraksi ciri pada citra ekspresi dan metode Convolutional Neural Network (CNN) sebagai prosesi klasifikasi emosi, dengan menggunakan data Facial Expression Recognition-2013 (FER-2013) dilakukan proses training dan testing untuk menghasilkan nilai akurasi dan pengenalan emosi wajah. Hasil pengujian akhir mendapatkan nilai akurasi pada metode PCA sebesar 59,375% dan nilai akurasi pada pengujian metode CNN sebesar 59,386%.

Sign in / Sign up

Export Citation Format

Share Document