scholarly journals Robust Multimodal Emotion Recognition from Conversation with Transformer-Based Crossmodality the title Fusion

Sensors ◽  
2021 ◽  
Vol 21 (14) ◽  
pp. 4913
Author(s):  
Baijun Xie ◽  
Mariia Sidulova ◽  
Chung Hyuk Park
Keyword(s):  
Emotion Recognition ◽  
Network Architecture ◽  
Emotional State ◽  
State Of The Art ◽  
State Recognition ◽  
Wide Range ◽  
Multimodal Emotion Recognition ◽  
Evaluating Methods ◽  
Audio Video ◽  
Evaluation Techniques

Decades of scientific research have been conducted on developing and evaluating methods for automated emotion recognition. With exponentially growing technology, there is a wide range of emerging applications that require emotional state recognition of the user. This paper investigates a robust approach for multimodal emotion recognition during a conversation. Three separate models for audio, video and text modalities are structured and fine-tuned on the MELD. In this paper, a transformer-based crossmodality fusion with the EmbraceNet architecture is employed to estimate the emotion. The proposed multimodal network architecture can achieve up to 65% accuracy, which significantly surpasses any of the unimodal models. We provide multiple evaluation techniques applied to our work to show that our model is robust and can even outperform the state-of-the-art models on the MELD.

scholarly journals EEG-based human emotion recognition using entropy as a feature extraction measure

2021 ◽  
Vol 8 (1) ◽  
Author(s):  
Pragati Patel ◽  
Raghunandan R ◽  
Ramesh Naidu Annavarapu
Keyword(s):  
Emotion Recognition ◽  
Emotional State ◽  
Computer Interface ◽  
Eeg Signal ◽  
Emotion Classification ◽  
Emotion Identification ◽  
Physiological Signal ◽  
Wide Range ◽  
Electrical Activities ◽  
The Brain

AbstractMany studies on brain–computer interface (BCI) have sought to understand the emotional state of the user to provide a reliable link between humans and machines. Advanced neuroimaging methods like electroencephalography (EEG) have enabled us to replicate and understand a wide range of human emotions more precisely. This physiological signal, i.e., EEG-based method is in stark comparison to traditional non-physiological signal-based methods and has been shown to perform better. EEG closely measures the electrical activities of the brain (a nonlinear system) and hence entropy proves to be an efficient feature in extracting meaningful information from raw brain waves. This review aims to give a brief summary of various entropy-based methods used for emotion classification hence providing insights into EEG-based emotion recognition. This study also reviews the current and future trends and discusses how emotion identification using entropy as a measure to extract features, can accomplish enhanced identification when using EEG signal.

Deep Learning Based Human Emotional State Recognition in a Video

2020 ◽  
Vol 12 (1) ◽  
pp. 51-59
Author(s):  
A. A. Moskvin ◽  
A.G. Shishkin
Keyword(s):  
Neural Networks ◽  
Emotion Recognition ◽  
Visual Information ◽  
Network Architecture ◽  
Neural Network Architecture ◽  
State Recognition ◽  
Audio Features ◽  
E Learning ◽  
Audio Information ◽  
Selection Of

Human emotions play significant role in everyday life. There are a lot of applications of automatic emotion recognition in medicine, e-learning, monitoring, marketing etc. In this paper the method and neural network architecture for real-time human emotion recognition by audio-visual data are proposed. To classify one of seven emotions, deep neural networks, namely, convolutional and recurrent neural networks are used. Visual information is represented by a sequence of 16 frames of 96 × 96 pixels, and audio information - by 140 features for each of a sequence of 37 temporal windows. To reduce the number of audio features autoencoder was used. Audio information in conjunction with visual one is shown to increase recognition accuracy up to 12%. The developed system being not demanding to be computing resources is dynamic in terms of selection of parameters, reducing or increasing the number of emotion classes, as well as the ability to easily add, accumulate and use information from other external devices for further improvement of classification accuracy. 

scholarly journals Human Emotional State Assessment Based on a Video Portrayal

2020 ◽  
Vol 13 (4) ◽  
pp. 4-24 ◽  
Author(s):  
V.A. Barabanschikov ◽  
E.V. Suvorova
Keyword(s):  
Emotion Recognition ◽  
Recognition Test ◽  
Emotional State ◽  
State Assessment ◽  
Identification Accuracy ◽  
Emotional Expressions ◽  
Emotional States ◽  
Wide Range ◽  
Similarities And Differences ◽  
Valence And Arousal

The article is devoted to the results of approbation of the Geneva Emotion Recognition Test (GERT), a Swiss method for assessing dynamic emotional states, on Russian sample. Identification accuracy and the categorical fields’ structure of emotional expressions of a “living” face are analysed. Similarities and differences in the perception of affective groups of dynamic emotions in the Russian and Swiss samples are considered. A number of patterns of recognition of multi-modal expressions with changes in valence and arousal of emotions are described. Differences in the perception of dynamics and statics of emotional expressions are revealed. GERT method confirmed it’s high potential for solving a wide range of academic and applied problems.

scholarly journals CpG Transformer for imputation of single-cell methylomes

2021 ◽  
Author(s):  
Gaetan De Waele ◽  
Jim Clauwaert ◽  
Gerben Menschaert ◽  
Willem Waegeman
Keyword(s):  
Single Cell ◽  
Network Architecture ◽  
State Of The Art ◽  
Time Budget ◽  
General Purpose ◽  
Biological Processes ◽  
Neural Network Architecture ◽  
Wide Range ◽  
Incomplete Coverage ◽  
Rapid Transfer

Motivation: The adoption of current single-cell DNA methylation sequencing protocols is hindered by incomplete coverage, outlining the need for effective imputation techniques. The task of imputing single-cell (methylation) data requires models to build an understanding of underlying biological processes. Current approaches compress intercellular methylation dependencies in some way and, hence, do not provide a general-purpose way of learning interactions between neighboring CpG sites both within- and between cells. Results: We adapt the transformer neural network architecture to operate on methylation matrices through the introduction of a novel 2D sliding window self-attention. The obtained CpG Transformer displays state-of-the-art performances on a wide range of scBS-seq and scRRBS-seq datasets. Furthermore, we demonstrate the interpretability of CpG Transformer and illustrate its rapid transfer learning properties, allowing practitioners to train models on new datasets with a limited computational and time budget. Availability and Implementation: CpG Transformer is freely available at https://github.com/gdewael/cpg-transformer.

scholarly journals Emotional State Recognition Using Facial Expression, Voice, and Physiological Signal

2018 ◽  
Vol 6 (1) ◽  
pp. 1-20
Author(s):  
Tahirou Djara ◽  
Abdoul Matine Ousmane ◽  
Antoine Vianou
Keyword(s):  
Signal Processing ◽  
Facial Expression ◽  
Emotion Recognition ◽  
Affective Computing ◽  
Emotional State ◽  
Processing Algorithm ◽  
State Recognition ◽  
Physiological Signal ◽  
Emotional Interaction

Emotion recognition is an important aspect of affective computing, one of whose aims is the study and development of behavioral and emotional interaction between human and machine. In this context, another important point concerns acquisition devices and signal processing tools which lead to an estimation of the emotional state of the user. This article presents a survey about concepts around emotion, multimodality in recognition, physiological activities and emotional induction, methods and tools for acquisition and signal processing with a focus on processing algorithm and their degree of reliability.

scholarly journals Emotional State Recognition Using Facial Expression, Voice, and Physiological Signal

2020 ◽  
pp. 1946-1967
Author(s):  
Tahirou Djara ◽  
Abdoul Matine Ousmane ◽  
Antoine Vianou
Keyword(s):  
Signal Processing ◽  
Facial Expression ◽  
Emotion Recognition ◽  
Affective Computing ◽  
Emotional State ◽  
Processing Algorithm ◽  
State Recognition ◽  
Physiological Signal ◽  
Emotional Interaction

Emotion recognition is an important aspect of affective computing, one of whose aims is the study and development of behavioral and emotional interaction between human and machine. In this context, another important point concerns acquisition devices and signal processing tools which lead to an estimation of the emotional state of the user. This article presents a survey about concepts around emotion, multimodality in recognition, physiological activities and emotional induction, methods and tools for acquisition and signal processing with a focus on processing algorithm and their degree of reliability.

scholarly journals Emotion Recognition from Skeletal Movements

Entropy ◽  
2019 ◽  
Vol 21 (7) ◽  
pp. 646 ◽  
Author(s):  
Tomasz Sapiński ◽  
Dorota Kamińska ◽  
Adam Pelikant ◽  
Gholamreza Anbarjafari
Keyword(s):  
Emotion Recognition ◽  
Emotional State ◽  
Body Movement ◽  
Experimental Results ◽  
Microsoft Kinect ◽  
Emotional States ◽  
Important Trend ◽  
Novel Method ◽  
Multimodal Emotion Recognition ◽  
Body Joints

Automatic emotion recognition has become an important trend in many artificial intelligence (AI) based applications and has been widely explored in recent years. Most research in the area of automated emotion recognition is based on facial expressions or speech signals. Although the influence of the emotional state on body movements is undeniable, this source of expression is still underestimated in automatic analysis. In this paper, we propose a novel method to recognise seven basic emotional states—namely, happy, sad, surprise, fear, anger, disgust and neutral—utilising body movement. We analyse motion capture data under seven basic emotional states recorded by professional actor/actresses using Microsoft Kinect v2 sensor. We propose a new representation of affective movements, based on sequences of body joints. The proposed algorithm creates a sequential model of affective movement based on low level features inferred from the spacial location and the orientation of joints within the tracked skeleton. In the experimental results, different deep neural networks were employed and compared to recognise the emotional state of the acquired motion sequences. The experimental results conducted in this work show the feasibility of automatic emotion recognition from sequences of body gestures, which can serve as an additional source of information in multimodal emotion recognition.

CLINICAL AND FORENSIC ASPECTS OF PHARMACOBEZOARS

2020 ◽  
Vol 12 ◽  
Author(s):  
Francisco Basílio ◽  
Ricardo Jorge Dinis-Oliveira
Keyword(s):  
State Of The Art ◽  
Signs And Symptoms ◽  
Immediate Release ◽  
Diagnosis And Treatment ◽  
Blood Concentrations ◽  
Forensic Practice ◽  
Wide Range ◽  
Therapeutic Doses ◽  
Complete History ◽  
Lethal Blood

Background: Pharmacobezoars are specific types of bezoars formed when medicines, such as tablets, suspensions, and/or drug delivery systems, aggregate and may cause death by occluding airways with tenacious material or by eluting drugs resulting in toxic or lethal blood concentrations. Objective: This work aims to fully review the state-of-the-art regarding pathophysiology, diagnosis, treatment and other relevant clinical and forensic features of pharmacobezoars. Results: patients of a wide range of ages and in both sexes present with signs and symptoms of intoxications or more commonly gastrointestinal obstructions. The exact mechanisms of pharmacobezoar formation are unknown but is likely multifactorial. The diagnosis and treatment depend on the gastrointestinal segment affected and should be personalized to the medication and the underlying factor. A good and complete history, physical examination, image tests, upper endoscopy and surgery through laparotomy of the lower tract are useful for diagnosis and treatment. Conclusion: Pharmacobezoars are rarely seen in clinical and forensic practice. They are related to controlled or immediate-release formulations, liquid or non-digestible substances, in normal or altered digestive motility/anatomy tract, and in overdoses or therapeutic doses, and should be suspected in the presence of risk factors or patients taking drugs which may form pharmacobezoars.

Computers in Geology - 25 Years of Progress

1994 ◽  
Keyword(s):  
Computer Modeling ◽  
Quantitative Methods ◽  
State Of The Art ◽  
International Association ◽  
Mathematical Geology ◽  
25Th Anniversary ◽  
The Earth ◽  
Wide Range ◽  
History Of ◽  
Mapping Techniques

This volume vividly demonstrates the importance and increasing breadth of quantitative methods in the earth sciences. With contributions from an international cast of leading practitioners, chapters cover a wide range of state-of-the-art methods and applications, including computer modeling and mapping techniques. Many chapters also contain reviews and extensive bibliographies which serve to make this an invaluable introduction to the entire field. In addition to its detailed presentations, the book includes chapters on the history of geomathematics and on R.G.V. Eigen, the "father" of mathematical geology. Written to commemorate the 25th anniversary of the International Association for Mathematical Geology, the book will be sought after by both practitioners and researchers in all branches of geology.

Sign in / Sign up

Export Citation Format

Share Document