Emotion recognition from audio-visual data using rule based decision level fusion

2016 ◽  
Author(s):  
Subhasmita Sahoo ◽  
Aurobinda Routray
Keyword(s):  
Emotion Recognition ◽  
Visual Data ◽  
Rule Based ◽  
Decision Level ◽  
Decision Level Fusion ◽  
Level Fusion

scholarly journals Emotion Recognition from Multimodal Physiological Signals for Emotion Aware Healthcare Systems

2020 ◽  
Vol 40 (2) ◽  
pp. 149-157 ◽  
Author(s):  
Değer Ayata ◽  
Yusuf Yaslan ◽  
Mustafa E. Kamasak
Keyword(s):  
Emotion Recognition ◽  
Healthcare Systems ◽  
Physiological Signals ◽  
Support Vector ◽  
Multiple Sources ◽  
Accuracy Rate ◽  
Decision Level ◽  
Decision Level Fusion ◽  
Level Fusion ◽  
Arousal And Valence

Abstract Purpose The purpose of this paper is to propose a novel emotion recognition algorithm from multimodal physiological signals for emotion aware healthcare systems. In this work, physiological signals are collected from a respiratory belt (RB), photoplethysmography (PPG), and fingertip temperature (FTT) sensors. These signals are used as their collection becomes easy with the advance in ergonomic wearable technologies. Methods Arousal and valence levels are recognized from the fused physiological signals using the relationship between physiological signals and emotions. This recognition is performed using various machine learning methods such as random forest, support vector machine and logistic regression. The performance of these methods is studied. Results Using decision level fusion, the accuracy improved from 69.86 to 73.08% for arousal, and from 69.53 to 72.18% for valence. Results indicate that using multiple sources of physiological signals and their fusion increases the accuracy rate of emotion recognition. Conclusion This study demonstrated a framework for emotion recognition using multimodal physiological signals from respiratory belt, photo plethysmography and fingertip temperature. It is shown that decision level fusion from multiple classifiers (one per signal source) improved the accuracy rate of emotion recognition both for arousal and valence dimensions.

scholarly journals A Multimodal Facial Emotion Recognition Framework through the Fusion of Speech with Visible and Infrared Images

2020 ◽  
Vol 4 (3) ◽  
pp. 46
Author(s):  
Mohammad Faridul Haque Siddiqui ◽  
Ahmad Y. Javaid
Keyword(s):  
Emotion Recognition ◽  
Final Decision ◽  
Infrared Images ◽  
Decision Level ◽  
Visible Images ◽  
Contemporary Work ◽  
Decision Level Fusion ◽  
Hybrid Fusion ◽  
Level Fusion ◽  
Fusion Approach

The exigency of emotion recognition is pushing the envelope for meticulous strategies of discerning actual emotions through the use of superior multimodal techniques. This work presents a multimodal automatic emotion recognition (AER) framework capable of differentiating between expressed emotions with high accuracy. The contribution involves implementing an ensemble-based approach for the AER through the fusion of visible images and infrared (IR) images with speech. The framework is implemented in two layers, where the first layer detects emotions using single modalities while the second layer combines the modalities and classifies emotions. Convolutional Neural Networks (CNN) have been used for feature extraction and classification. A hybrid fusion approach comprising early (feature-level) and late (decision-level) fusion, was applied to combine the features and the decisions at different stages. The output of the CNN trained with voice samples of the RAVDESS database was combined with the image classifier’s output using decision-level fusion to obtain the final decision. An accuracy of 86.36% and similar recall (0.86), precision (0.88), and f-measure (0.87) scores were obtained. A comparison with contemporary work endorsed the competitiveness of the framework with the rationale for exclusivity in attaining this accuracy in wild backgrounds and light-invariant conditions.

scholarly journals Feature and Decision Level Fusion in Children Multimodal Biometrics

2020 ◽  
Vol 8 (5) ◽  
pp. 2522-2527
Keyword(s):  
Nearest Neighbor ◽  
Principal Component ◽  
Identification Accuracy ◽  
Support Vector ◽  
Biometric System ◽  
K Nearest Neighbor ◽  
Decision Level ◽  
Multimodal Biometric System ◽  
Decision Level Fusion ◽  
Level Fusion

In this paper, we design method for recognition of fingerprint and IRIS using feature level fusion and decision level fusion in Children multimodal biometric system. Initially, Histogram of Gradients (HOG), Gabour and Maximum filter response are extracted from both the domains of fingerprint and IRIS and considered for identification accuracy. The combination of feature vector of all the possible features is recommended by biometrics traits of fusion. For fusion vector the Principal Component Analysis (PCA) is used to select features. The reduced features are fed into fusion classifier of K-Nearest Neighbor (KNN), Support Vector Machine (SVM), Navie Bayes(NB). For children multimodal biometric system the suitable combination of features and fusion classifiers is identified. The experimentation conducted on children’s fingerprint and IRIS database and results reveal that fusion combination outperforms individual. In addition the proposed model advances the unimodal biometrics system.

Sign in / Sign up

Export Citation Format

Share Document