scholarly journals Driver Facial Expression Analysis Using LFA-CRNN-Based Feature Extraction for Health-Risk Decisions

2020 ◽  
Vol 10 (8) ◽  
pp. 2956 ◽  
Author(s):  
Chang-Min Kim ◽  
Ellen J. Hong ◽  
Kyungyong Chung ◽  
Roy C. Park
Keyword(s):  
Feature Extraction ◽  
Facial Expression ◽  
Health Risk ◽  
Everyday Life ◽  
Facial Expressions ◽  
Health Risks ◽  
Line Segment ◽  
Emotional State ◽  
Feature Analysis ◽  
Facial Expression Analysis

As people communicate with each other, they use gestures and facial expressions as a means to convey and understand emotional state. Non-verbal means of communication are essential to understanding, based on external clues to a person’s emotional state. Recently, active studies have been conducted on the lifecare service of analyzing users’ facial expressions. Yet, rather than a service necessary for everyday life, the service is currently provided only for health care centers or certain medical institutions. It is necessary to conduct studies to prevent accidents that suddenly occur in everyday life and to cope with emergencies. Thus, we propose facial expression analysis using line-segment feature analysis-convolutional recurrent neural network (LFA-CRNN) feature extraction for health-risk assessments of drivers. The purpose of such an analysis is to manage and monitor patients with chronic diseases who are rapidly increasing in number. To prevent automobile accidents and to respond to emergency situations due to acute diseases, we propose a service that monitors a driver’s facial expressions to assess health risks and alert the driver to risk-related matters while driving. To identify health risks, deep learning technology is used to recognize expressions of pain and to determine if a person is in pain while driving. Since the amount of input-image data is large, analyzing facial expressions accurately is difficult for a process with limited resources while providing the service on a real-time basis. Accordingly, a line-segment feature analysis algorithm is proposed to reduce the amount of data, and the LFA-CRNN model was designed for this purpose. Through this model, the severity of a driver’s pain is classified into one of nine types. The LFA-CRNN model consists of one convolution layer that is reshaped and delivered into two bidirectional gated recurrent unit layers. Finally, biometric data are classified through softmax. In addition, to evaluate the performance of LFA-CRNN, the performance was compared through the CRNN and AlexNet Models based on the University of Northern British Columbia and McMaster University (UNBC-McMaster) database.

Automatic Facial Expression Recognition by Facial Parts Location with Boosted-LBP

2013 ◽  
pp. 42-55
Author(s):  
Yi Ji ◽  
Khalid Idrissi
Keyword(s):  
Feature Extraction ◽  
Facial Expression ◽  
Facial Expressions ◽  
Facial Expression Recognition ◽  
Recognition System ◽  
Video Sequences ◽  
Polynomial Kernel ◽  
Expression Recognition ◽  
Emotional States ◽  
Small Set

This paper proposes an automatic facial expression recognition system, which uses new methods in both face detection and feature extraction. In this system, considering that facial expressions are related to a small set of muscles and limited ranges of motions, the facial expressions are recognized by these changes in video sequences. First, the differences between neutral and emotional states are detected. Faces can be automatically located from changing facial organs. Then, LBP features are applied and AdaBoost is used to find the most important features for each expression on essential facial parts. At last, SVM with polynomial kernel is used to classify expressions. The method is evaluated on JAFFE and MMI databases. The performances are better than other automatic or manual annotated systems.

Spontaneous Facial Expression Analysis and Synthesis for Interactive Facial Animation

2011 ◽  
pp. 20-37
Author(s):  
Yongmian Zhang ◽  
Jixu Chen ◽  
Yan Tong ◽  
Qiang Ji
Keyword(s):  
Facial Expression ◽  
Bayesian Network ◽  
Facial Expressions ◽  
Expression Analysis ◽  
Facial Animation ◽  
Head Movements ◽  
Perceptual Quality ◽  
Facial Action ◽  
Facial Expression Analysis ◽  
Analysis And Synthesis

This chapter describes a probabilistic framework for faithful reproduction of spontaneous facial expressions on a synthetic face model in a real time interactive application. The framework consists of a coupled Bayesian network (BN) to unify the facial expression analysis and synthesis into one coherent structure. At the analysis end, we cast the facial action coding system (FACS) into a dynamic Bayesian network (DBN) to capture relationships between facial expressions and the facial motions as well as their uncertainties and dynamics. The observations fed into the DBN facial expression model are measurements of facial action units (AUs) generated by an AU model. Also implemented by a DBN, the AU model captures the rigid head movements and nonrigid facial muscular movements of a spontaneous facial expression. At the synthesizer, a static BN reconstructs the Facial Animation Parameters (FAPs) and their intensity through the top-down inference according to the current state of facial expression and pose information output by the analysis end. The two BNs are connected statically through a data stream link. The novelty of using the coupled BN brings about several benefits. First, a facial expression is inferred through both spatial and temporal inference so that the perceptual quality of animation is less affected by the misdetection of facial features. Second, more realistic looking facial expressions can be reproduced by modeling the dynamics of human expressions in facial expression analysis. Third, very low bitrate (9 bytes per frame) in data transmission can be achieved.

Affective Tutoring System for Better Learning

2011 ◽  
pp. 134-150 ◽  
Author(s):  
Abdolhossein Sarrafzadeh ◽  
Samuel T.V. Alexander ◽  
Jamshid Shanbehzadeh
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Recognition System ◽  
Expression Recognition ◽  
Tutoring System ◽  
Tutoring Systems ◽  
System A ◽  
Facial Expression Analysis ◽  
Animated Agent ◽  
Case Based

Intelligent tutoring systems (ITS) are still not as effective as one-on-one human tutoring. The next generation of intelligent tutors are expected to be able to take into account the emotional state of students. This paper presents research on the development of an Affective Tutoring System (ATS). The system called “Easy with Eve” adapts to students via a lifelike animated agent who is able to detect student emotion through facial expression analysis, and can display emotion herself. Eve’s adaptations are guided by a case-based method for adapting to student states; this method uses data that was generated by an observational study of human tutors. This paper presents an analysis of facial expressions of students engaged in learning with human tutors and how a facial expression recognition system, a life like agent and a case based system based on this analysis have been integrated to develop an ATS for mathematics.

Research on the Facial Expression Feature Extraction of Facial Expression Recognition Based on MATLAB

2014 ◽  
Vol 1049-1050 ◽  
pp. 1522-1525
Author(s):  
Wang Ju ◽  
Ding Rui ◽  
Chun Yan Nie
Keyword(s):  
Pattern Recognition ◽  
Feature Extraction ◽  
Facial Expression ◽  
Interpersonal Communication ◽  
Facial Expressions ◽  
Facial Expression Recognition ◽  
Facial Feature ◽  
Expression Recognition ◽  
Information Communication ◽  
Facial Feature Extraction

In such a developed day of information communication, communication is an important essential way of interpersonal communication. As a carrier of information, expression is rich in human behavior information. Facial expression recognition is a combination of many fields, but also a new topic in the field of pattern recognition. This paper mainly studied the facial feature extraction based on MATLAB, by MATLAB software, extracting the expression features through a large number of facial expressions, which can be divided into different facial expressions more accurate classification .

scholarly journals Facial Expression Analysis using Convolutional Neural Networks

2020 ◽  
Vol 9 (4) ◽  
pp. 1142-1146
Keyword(s):  
Neural Networks ◽  
Facial Expression ◽  
Real Time ◽  
Facial Expressions ◽  
Convolutional Neural Networks ◽  
Expression Analysis ◽  
Facial Expression Recognition ◽  
Expression Recognition ◽  
Real Time System ◽  
Facial Expression Analysis

Human feelings are mental conditions of sentiments that emerge immediately as opposed to cognitive exertion. Some of the basic feelings are happy, angry, neutral, sad and surprise. These internal feelings of a person are reflected on the face as Facial Expressions. This paper presents a novel methodology for Facial Expression Analysis which will aid to develop a facial expression recognition system. This system can be used in real time to classify five basic emotions. The recognition of facial expressions is important because of its applications in many domains such as artificial intelligence, security and robotics. Many different approaches can be used to overcome the problems of Facial Expression Recognition (FER) but the best suited technique for automated FER is Convolutional Neural Networks(CNN). Thus, a novel CNN architecture is proposed and a combination of multiple datasets such as FER2013, FER+, JAFFE and CK+ is used for training and testing. This helps to improve the accuracy and develop a robust real time system. The proposed methodology confers quite good results and the obtained accuracy may give encouragement and offer support to researchers to build better models for Automated Facial Expression Recognition systems.

scholarly journals Facial Expression Recognition Using Fast Walidlet Hybrid Transform

2020 ◽  
pp. 59-69
Author(s):  
Walid Mahmod ◽  
Jane Stephan ◽  
Anmar Razzak
Keyword(s):  
Feature Extraction ◽  
Facial Expression ◽  
Facial Expressions ◽  
Facial Expression Recognition ◽  
Good Accuracy ◽  
Expression Recognition ◽  
Coding System ◽  
Facial Action Coding System ◽  
Artificial Intelligence Research ◽  
Multiwavelet Transform

Automatic analysis of facial expressions is rapidly becoming an area of intense interest in computer vision and artificial intelligence research communities. In this paper an approach is presented for facial expression recognition of the six basic prototype expressions (i.e., joy, surprise, anger, sadness, fear, and disgust) based on Facial Action Coding System (FACS). The approach is attempting to utilize a combination of different transforms (Walid let hybrid transform); they consist of Fast Fourier Transform; Radon transform and Multiwavelet transform for the feature extraction. Korhonen Self Organizing Feature Map (SOFM) then used for patterns clustering based on the features obtained from the hybrid transform above. The result shows that the method has very good accuracy in facial expression recognition. However, the proposed method has many promising features that make it interesting. The approach provides a new method of feature extraction in which overcome the problem of the illumination, faces that varies from one individual to another quite considerably due to different age, ethnicity, gender and cosmetic also it does not require a precise normalization and lighting equalization. An average clustering accuracy of 94.8% is achieved for six basic expressions, where different databases had been used for the test of the method.

scholarly journals Facial and Body Feature Extraction for Emotionally-Rich HCI

2011 ◽  
pp. 2183-2192
Author(s):  
Kostas Karpouzis ◽  
Athanasios Drosopoulos ◽  
Spiros Ioannou ◽  
Amaryllis Raouzaiou ◽  
Nicolas Tsapatsoulis ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Computer Vision ◽  
Feature Extraction ◽  
Facial Expressions ◽  
Emotional State ◽  
Different Cultures ◽  
Machine Interaction

Emotionally-aware Man-Machine Interaction (MMI) systems are presently at the forefront of interest of the computer vision and artificial intelligence communities, since they give the opportunity to less technology-aware people to use computers more efficiently, overcoming fears and preconceptions. Most emotion-related facial and body gestures are considered to be universal, in the sense that they are recognized along different cultures; therefore, the introduction of an “emotional dictionary” that includes descriptions and perceived meanings of facial expressions and body gestures, so as to help infer the likely emotional state of a specific user, can enhance the affective nature of MMI applications (Picard, 2000).

scholarly journals Facial and Body Feature Extraction for Emotionally-Rich HCI

2005 ◽  
pp. 1180-1185
Author(s):  
Kostas Karpouzis ◽  
Athanasios Drosopoulos ◽  
Spiros Ioannou ◽  
Amaryllis Raouzaiou ◽  
Nicolas Tsapatsoulis ◽  
...  
Keyword(s):  
Artificial Intelligence ◽  
Computer Vision ◽  
Feature Extraction ◽  
Facial Expressions ◽  
Emotional State ◽  
Different Cultures ◽  
Machine Interaction

Emotionally-aware Man-Machine Interaction (MMI) systems are presently at the forefront of interest of the computer vision and artificial intelligence communities, since they give the opportunity to less technology-aware people to use computers more efficiently, overcoming fears and preconceptions. Most emotion-related facial and body gestures are considered to be universal, in the sense that they are recognized along different cultures; therefore, the introduction of an “emotional dictionary” that includes descriptions and perceived meanings of facial expressions and body gestures, so as to help infer the likely emotional state of a specific user, can enhance the affective nature of MMI applications (Picard, 2000).

scholarly journals Predicting Group Contribution Behaviour in a Public Goods Game from Face-to-Face Communication

Sensors ◽  
2019 ◽  
Vol 19 (12) ◽  
pp. 2786 ◽  
Author(s):  
Ehsan Othman ◽  
Frerk Saxen ◽  
Dmitri Bershadskyy ◽  
Philipp Werner ◽  
Ayoub Al-Hamadi ◽  
...  
Keyword(s):  
Facial Expression ◽  
Public Goods ◽  
Facial Expressions ◽  
Free Riding ◽  
Random Forest Classification ◽  
Face To Face ◽  
Public Goods Games ◽  
Facial Expression Analysis ◽  
Forest Classification ◽  
Theoretical Predictions

Experimental economic laboratories run many studies to test theoretical predictions with actual human behaviour, including public goods games. With this experiment, participants in a group have the option to invest money in a public account or to keep it. All the invested money is multiplied and then evenly distributed. This structure incentivizes free riding, resulting in contributions to the public goods declining over time. Face-to-face Communication (FFC) diminishes free riding and thus positively affects contribution behaviour, but the question of how has remained mostly unknown. In this paper, we investigate two communication channels, aiming to explain what promotes cooperation and discourages free riding. Firstly, the facial expressions of the group in the 3-minute FFC videos are automatically analysed to predict the group behaviour towards the end of the game. The proposed automatic facial expressions analysis approach uses a new group activity descriptor and utilises random forest classification. Secondly, the contents of FFC are investigated by categorising strategy-relevant topics and using meta-data. The results show that it is possible to predict whether the group will fully contribute to the end of the games based on facial expression data from three minutes of FFC, but deeper understanding requires a larger dataset. Facial expression analysis and content analysis found that FFC and talking until the very end had a significant, positive effect on the contributions.

Facial Action Recognition in 2D and 3D

2014 ◽  
pp. 167-186 ◽  
Author(s):  
Michel Valstar ◽  
Stefanos Zafeiriou ◽  
Maja Pantic
Keyword(s):  
Facial Expression ◽  
Facial Expressions ◽  
Expression Analysis ◽  
Coding System ◽  
Recording Equipment ◽  
Facial Action ◽  
Facial Expression Analysis ◽  
Challenges And Opportunities ◽  
Almost All ◽  
2D And 3D

Automatic Facial Expression Analysis systems have come a long way since the earliest approaches in the early 1970s. We are now at a point where the first systems are commercially applied, most notably smile detectors included in digital cameras. As one of the most comprehensive and objective ways to describe facial expressions, the Facial Action Coding System (FACS) has received significant and sustained attention within the field. Over the past 30 years, psychologists and neuroscientists have conducted extensive research on various aspects of human behaviour using facial expression analysis coded in terms of FACS. Automating FACS coding would make this research faster and more widely applicable, opening up new avenues to understanding how we communicate through facial expressions. Mainly due to the cost effectiveness of existing recording equipment, until recently almost all work conducted in this area involves 2D imagery, despite their inherent problems relating to pose and illumination variations. In order to deal with these problems, 3D recordings are increasingly used in expression analysis research. In this chapter, the authors give an overview of 2D and 3D FACS recognition, and summarise current challenges and opportunities.

Sign in / Sign up

Export Citation Format

Share Document