scholarly journals Towards an Architecture Model for Emotion Recognition in Interactive Systems: Application to a Ballet Dance Show

2009 ◽  
Author(s):  
Alexis Clay ◽  
Nadine Couture ◽  
Laurence Nigay
Keyword(s):  
Software Engineering ◽  
Emotion Recognition ◽  
Affective Computing ◽  
Interactive Systems ◽  
Point Of View ◽  
Architecture Model ◽  
Multimodal Emotion Recognition

In the context of the very dynamic and challenging domain of affective computing, we adopt a software engineering point of view on emotion recognition in interactive systems. Our goal is threefold: first, developing an architecture model for emotion recognition. This architecture model emphasizes multimodality and reusability. Second, developing a prototype based on this architecture model. For this prototype we focus on gesture-based emotion recognition. And third, using this prototype for augmenting a ballet dance show. We hence describe an overview of our work so far, from the design of a flexible and multimodal emotion recognition architecture model, to a presentation of a gesture-based emotion recognition prototype based on this model, to a prototype that augments a ballet stage, taking emotions as inputs.

scholarly journals Emotion Recognition for Human-Robot Interaction: Recent Advances and Future Perspectives

2020 ◽  
Vol 7 ◽  
Author(s):  
Matteo Spezialetti ◽  
Giuseppe Placidi ◽  
Silvia Rossi
Keyword(s):  
Emotion Recognition ◽  
Affective Computing ◽  
Brain Activity ◽  
Human Robot Interaction ◽  
Point Of View ◽  
Robot Interaction ◽  
Human Machine Interaction ◽  
Recent Advances ◽  
Critical Issues ◽  
Machine Interaction

A fascinating challenge in the field of human–robot interaction is the possibility to endow robots with emotional intelligence in order to make the interaction more intuitive, genuine, and natural. To achieve this, a critical point is the capability of the robot to infer and interpret human emotions. Emotion recognition has been widely explored in the broader fields of human–machine interaction and affective computing. Here, we report recent advances in emotion recognition, with particular regard to the human–robot interaction context. Our aim is to review the state of the art of currently adopted emotional models, interaction modalities, and classification strategies and offer our point of view on future developments and critical issues. We focus on facial expressions, body poses and kinematics, voice, brain activity, and peripheral physiological responses, also providing a list of available datasets containing data from these modalities.

scholarly journals Emotion Recognition of Chinese Paintings at the Thirteenth National Exhibition of Fines Arts in China Based on Advanced Affective Computing

2021 ◽  
Vol 12 ◽  
Author(s):  
Jing Li ◽  
Dongliang Chen ◽  
Ning Yu ◽  
Ziping Zhao ◽  
Zhihan Lv
Keyword(s):  
Neural Network ◽  
Emotion Recognition ◽  
Affective Computing ◽  
Chinese Painting ◽  
Training Set ◽  
Chi Square ◽  
Test Set ◽  
Chi Square Test ◽  
Multimodal Emotion Recognition ◽  
Traditional Chinese Painting

Today, with the rapid development of economic level, people’s esthetic requirements are also rising, they have a deeper emotional understanding of art, and the voice of their traditional art and culture is becoming higher. The study expects to explore the performance of advanced affective computing in the recognition and analysis of emotional features of Chinese paintings at the 13th National Exhibition of Fines Arts. Aiming at the problem of “semantic gap” in the emotion recognition task of images such as traditional Chinese painting, the study selects the AlexNet algorithm based on convolutional neural network (CNN), and further improves the AlexNet algorithm. Meanwhile, the study adds chi square test to solve the problems of data redundancy and noise in various modes such as Chinese painting. Moreover, the study designs a multimodal emotion recognition model of Chinese painting based on improved AlexNet neural network and chi square test. Finally, the performance of the model is verified by simulation with Chinese painting in the 13th National Exhibition of Fines Arts as the data source. The proposed algorithm is compared with Long Short-Term Memory (LSTM), CNN, Recurrent Neural Network (RNN), AlexNet, and Deep Neural Network (DNN) algorithms from the training set and test set, respectively, The emotion recognition accuracy of the proposed algorithm reaches 92.23 and 97.11% in the training set and test set, respectively, the training time is stable at about 54.97 s, and the test time is stable at about 23.74 s. In addition, the analysis of the acceleration efficiency of each algorithm shows that the improved AlexNet algorithm is suitable for processing a large amount of brain image data, and the acceleration ratio is also higher than other algorithms. And the efficiency in the test set scenario is slightly better than that in the training set scenario. On the premise of ensuring the error, the multimodal emotion recognition model of Chinese painting can achieve high accuracy and obvious acceleration effect. More importantly, the emotion recognition and analysis effect of traditional Chinese painting is the best, which can provide an experimental basis for the digital understanding and management of emotion of quintessence.

scholarly journals Advances in Multimodal Emotion Recognition Based on Brain–Computer Interfaces

2020 ◽  
Vol 10 (10) ◽  
pp. 687 ◽  
Author(s):  
Zhipeng He ◽  
Zina Li ◽  
Fuzhou Yang ◽  
Lei Wang ◽  
Jingcong Li ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Affective Computing ◽  
Experimental Results ◽  
Brain Computer Interfaces ◽  
Continuous Development ◽  
Research Directions ◽  
Sensory Stimuli ◽  
Computer Interfaces ◽  
Brain Signals ◽  
Multimodal Emotion Recognition

With the continuous development of portable noninvasive human sensor technologies such as brain–computer interfaces (BCI), multimodal emotion recognition has attracted increasing attention in the area of affective computing. This paper primarily discusses the progress of research into multimodal emotion recognition based on BCI and reviews three types of multimodal affective BCI (aBCI): aBCI based on a combination of behavior and brain signals, aBCI based on various hybrid neurophysiology modalities and aBCI based on heterogeneous sensory stimuli. For each type of aBCI, we further review several representative multimodal aBCI systems, including their design principles, paradigms, algorithms, experimental results and corresponding advantages. Finally, we identify several important issues and research directions for multimodal emotion recognition based on BCI.

scholarly journals Emotion Mining: from Unimodal to Multimodal Approaches

2021 ◽  
pp. 143-158
Author(s):  
Chiara Zucco ◽  
Barbara Calabrese ◽  
Mario Cannataro
Keyword(s):  
Social Media ◽  
Deep Learning ◽  
Emotion Recognition ◽  
Affective Computing ◽  
Large Datasets ◽  
Social Media Platforms ◽  
Basic Concepts ◽  
Multimodal Emotion Recognition ◽  
Learning Architectures ◽  
Emotion Mining

AbstractIn the last decade, Sentiment Analysis and Affective Computing have found applications in different domains. In particular, the interest of extracting emotions in healthcare is demonstrated by the various applications which encompass patient monitoring and adverse events prediction. Thanks to the availability of large datasets, most of which are extracted from social media platforms, several techniques for extracting emotion and opinion from different modalities have been proposed, using both unimodal and multimodal approaches. After introducing the basic concepts related to emotion theories, mainly borrowed from social sciences, the present work reviews three basic modalities used in emotion recognition, i.e. textual, audio and video, presenting for each of these i) some basic methodologies, ii) some among the widely used datasets for the training of supervised algorithms and iii) briefly discussing some deep Learning architectures. Furthermore, the paper outlines the challenges and existing resources to perform a multimodal emotion recognition which may improve performances by combining at least two unimodal approaches. architecture to perform multimodal emotion recognition.

Rationale and Design Considerations for a Semantic Mediator in Health Information Systems

1998 ◽  
Vol 37 (04/05) ◽  
pp. 518-526 ◽  
Author(s):  
D. Sauquet ◽  
M.-C. Jaulent ◽  
E. Zapletal ◽  
M. Lavril ◽  
P. Degoulet
Keyword(s):  
Information Systems ◽  
Software Engineering ◽  
Health Information ◽  
Rapid Development ◽  
Heterogeneous Systems ◽  
Health Information Systems ◽  
Semantic Interoperability ◽  
Point Of View ◽  
Current State ◽  
Engineering Environment

AbstractRapid development of community health information networks raises the issue of semantic interoperability between distributed and heterogeneous systems. Indeed, operational health information systems originate from heterogeneous teams of independent developers and have to cooperate in order to exchange data and services. A good cooperation is based on a good understanding of the messages exchanged between the systems. The main issue of semantic interoperability is to ensure that the exchange is not only possible but also meaningful. The main objective of this paper is to analyze semantic interoperability from a software engineering point of view. It describes the principles for the design of a semantic mediator (SM) in the framework of a distributed object manager (DOM). The mediator is itself a component that should allow the exchange of messages independently of languages and platforms. The functional architecture of such a SM is detailed. These principles have been partly applied in the context of the HEllOS object-oriented software engineering environment. The resulting service components are presented with their current state of achievement.

Multimodal emotion recognition with hierarchical memory networks

2021 ◽  
Vol 25 (4) ◽  
pp. 1031-1045
Author(s):  
Helang Lai ◽  
Keke Wu ◽  
Lingli Li
Keyword(s):  
Emotion Recognition ◽  
The Self ◽  
Global Memory ◽  
Human Computer Interactions ◽  
Accuracy Improvement ◽  
Local Memory ◽  
Hierarchical Memory ◽  
Multimodal Emotion Recognition ◽  
Novel Model ◽  
Computer Interactions

Emotion recognition in conversations is crucial as there is an urgent need to improve the overall experience of human-computer interactions. A promising improvement in this field is to develop a model that can effectively extract adequate contexts of a test utterance. We introduce a novel model, termed hierarchical memory networks (HMN), to address the issues of recognizing utterance level emotions. HMN divides the contexts into different aspects and employs different step lengths to represent the weights of these aspects. To model the self dependencies, HMN takes independent local memory networks to model these aspects. Further, to capture the interpersonal dependencies, HMN employs global memory networks to integrate the local outputs into global storages. Such storages can generate contextual summaries and help to find the emotional dependent utterance that is most relevant to the test utterance. With an attention-based multi-hops scheme, these storages are then merged with the test utterance using an addition operation in the iterations. Experiments on the IEMOCAP dataset show our model outperforms the compared methods with accuracy improvement.

scholarly journals Investigating the Relationship Between Emotion Recognition Software and Usability Metrics

i-com ◽  
2020 ◽  
Vol 19 (2) ◽  
pp. 139-151
Author(s):  
Thomas Schmidt ◽  
Miriam Schlindwein ◽  
Katharina Lichtner ◽  
Christian Wolff
Keyword(s):  
Emotion Recognition ◽  
Regression Models ◽  
Affective Computing ◽  
Large Scale ◽  
General Purpose ◽  
Thinking Aloud ◽  
Continue Research ◽  
Usability Metrics ◽  
Recognition Software ◽  
The Relationship

AbstractDue to progress in affective computing, various forms of general purpose sentiment/emotion recognition software have become available. However, the application of such tools in usability engineering (UE) for measuring the emotional state of participants is rarely employed. We investigate if the application of sentiment/emotion recognition software is beneficial for gathering objective and intuitive data that can predict usability similar to traditional usability metrics. We present the results of a UE project examining this question for the three modalities text, speech and face. We perform a large scale usability test (N = 125) with a counterbalanced within-subject design with two websites of varying usability. We have identified a weak but significant correlation between text-based sentiment analysis on the text acquired via thinking aloud and SUS scores as well as a weak positive correlation between the proportion of neutrality in users’ voice and SUS scores. However, for the majority of the output of emotion recognition software, we could not find any significant results. Emotion metrics could not be used to successfully differentiate between two websites of varying usability. Regression models, either unimodal or multimodal could not predict usability metrics. We discuss reasons for these results and how to continue research with more sophisticated methods.

scholarly journals Affective Computing on Machine Learning-Based Emotion Recognition Using a Self-Made EEG Device

Sensors ◽  
2021 ◽  
Vol 21 (15) ◽  
pp. 5135
Author(s):  
Ngoc-Dau Mai ◽  
Boon-Giin Lee ◽  
Wan-Young Chung
Keyword(s):  
Machine Learning ◽  
Emotion Recognition ◽  
Temporal Lobe ◽  
Affective Computing ◽  
Electrode Placement ◽  
Entropy Measure ◽  
Support Vector ◽  
Emotion Classification ◽  
Eeg Signals ◽  
Computing Method

In this research, we develop an affective computing method based on machine learning for emotion recognition using a wireless protocol and a wearable electroencephalography (EEG) custom-designed device. The system collects EEG signals using an eight-electrode placement on the scalp; two of these electrodes were placed in the frontal lobe, and the other six electrodes were placed in the temporal lobe. We performed experiments on eight subjects while they watched emotive videos. Six entropy measures were employed for extracting suitable features from the EEG signals. Next, we evaluated our proposed models using three popular classifiers: a support vector machine (SVM), multi-layer perceptron (MLP), and one-dimensional convolutional neural network (1D-CNN) for emotion classification; both subject-dependent and subject-independent strategies were used. Our experiment results showed that the highest average accuracies achieved in the subject-dependent and subject-independent cases were 85.81% and 78.52%, respectively; these accuracies were achieved using a combination of the sample entropy measure and 1D-CNN. Moreover, our study investigates the T8 position (above the right ear) in the temporal lobe as the most critical channel among the proposed measurement positions for emotion classification through electrode selection. Our results prove the feasibility and efficiency of our proposed EEG-based affective computing method for emotion recognition in real-world applications.

Sign in / Sign up

Export Citation Format

Share Document