Multi-Modal Speech Emotion Recognition Using Speech Embeddings and Audio Features

2019 ◽  
Author(s):  
Krishna D N ◽  
Sai Sumith Reddy
Keyword(s):  
Emotion Recognition ◽  
Speech Emotion Recognition ◽  
Audio Features

scholarly journals Utterance Level Feature Aggregation with Deep Metric Learning for Speech Emotion Recognition

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 4233
Author(s):  
Bogdan Mocanu ◽  
Ruxandra Tapu ◽  
Titus Zaharia
Keyword(s):  
Emotion Recognition ◽  
Loss Function ◽  
State Of The Art ◽  
Disease Diagnosis ◽  
Data Representation ◽  
Speech Emotion Recognition ◽  
Audio Features ◽  
Global Accuracy ◽  
Space Data ◽  
Art Techniques

Emotion is a form of high-level paralinguistic information that is intrinsically conveyed by human speech. Automatic speech emotion recognition is an essential challenge for various applications; including mental disease diagnosis; audio surveillance; human behavior understanding; e-learning and human–machine/robot interaction. In this paper, we introduce a novel speech emotion recognition method, based on the Squeeze and Excitation ResNet (SE-ResNet) model and fed with spectrogram inputs. In order to overcome the limitations of the state-of-the-art techniques, which fail in providing a robust feature representation at the utterance level, the CNN architecture is extended with a trainable discriminative GhostVLAD clustering layer that aggregates the audio features into compact, single-utterance vector representation. In addition, an end-to-end neural embedding approach is introduced, based on an emotionally constrained triplet loss function. The loss function integrates the relations between the various emotional patterns and thus improves the latent space data representation. The proposed methodology achieves 83.35% and 64.92% global accuracy rates on the RAVDESS and CREMA-D publicly available datasets, respectively. When compared with the results provided by human observers, the gains in global accuracy scores are superior to 24%. Finally, the objective comparative evaluation with state-of-the-art techniques demonstrates accuracy gains of more than 3%.

scholarly journals Deep Visual Attributes vs. Hand-Crafted Audio Features on Multidomain Speech Emotion Recognition

Computation ◽  
2017 ◽  
Vol 5 (4) ◽  
pp. 26 ◽  
Author(s):  
Michalis Papakostas ◽  
Evaggelos Spyrou ◽  
Theodoros Giannakopoulos ◽  
Giorgos Siantikos ◽  
Dimitrios Sgouropoulos ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Speech Emotion Recognition ◽  
Visual Attributes ◽  
Audio Features

The Robust Spectral Audio Features for Speech Emotion Recognition

2019 ◽  
Vol 13 (5) ◽  
pp. 867-870
Author(s):  
Aisultan, Kanat, Nazerke, Rakhima Shoiynbek, Kozhakhmet, Sultanova, Zhumaliyeva
Keyword(s):  
Emotion Recognition ◽  
Speech Emotion Recognition ◽  
Audio Features

scholarly journals 3D CNN-Based Speech Emotion Recognition Using K-Means Clustering and Spectrograms

Entropy ◽  
2019 ◽  
Vol 21 (5) ◽  
pp. 479 ◽  
Author(s):  
Noushin Hajarolasvadi ◽  
Hasan Demirel
Keyword(s):  
Emotion Recognition ◽  
Speech Signal ◽  
Expressed Emotion ◽  
Audio Signal ◽  
Research Direction ◽  
Recognition System ◽  
Speech Emotion Recognition ◽  
Mel Frequency Cepstral Coefficients ◽  
Audio Features ◽  
3D Cnn

Detecting human intentions and emotions helps improve human–robot interactions. Emotion recognition has been a challenging research direction in the past decade. This paper proposes an emotion recognition system based on analysis of speech signals. Firstly, we split each speech signal into overlapping frames of the same length. Next, we extract an 88-dimensional vector of audio features including Mel Frequency Cepstral Coefficients (MFCC), pitch, and intensity for each of the respective frames. In parallel, the spectrogram of each frame is generated. In the final preprocessing step, by applying k-means clustering on the extracted features of all frames of each audio signal, we select k most discriminant frames, namely keyframes, to summarize the speech signal. Then, the sequence of the corresponding spectrograms of keyframes is encapsulated in a 3D tensor. These tensors are used to train and test a 3D Convolutional Neural network using a 10-fold cross-validation approach. The proposed 3D CNN has two convolutional layers and one fully connected layer. Experiments are conducted on the Surrey Audio-Visual Expressed Emotion (SAVEE), Ryerson Multimedia Laboratory (RML), and eNTERFACE’05 databases. The results are superior to the state-of-the-art methods reported in the literature.

Speech Emotion Recognition Based on Sparse Representation

2013 ◽  
Vol 38 (4) ◽  
pp. 465-470 ◽  
Author(s):  
Jingjie Yan ◽  
Xiaolan Wang ◽  
Weiyi Gu ◽  
LiLi Ma
Keyword(s):  
Dimensionality Reduction ◽  
Emotion Recognition ◽  
Least Squares ◽  
Partial Least Squares ◽  
Partial Least Squares Regression ◽  
Speech Emotion Recognition ◽  
Least Squares Regression ◽  
Computer Science Pedagogy ◽  
Reduction Methods ◽  
Analysis Computer

Abstract Speech emotion recognition is deemed to be a meaningful and intractable issue among a number of do- mains comprising sentiment analysis, computer science, pedagogy, and so on. In this study, we investigate speech emotion recognition based on sparse partial least squares regression (SPLSR) approach in depth. We make use of the sparse partial least squares regression method to implement the feature selection and dimensionality reduction on the whole acquired speech emotion features. By the means of exploiting the SPLSR method, the component parts of those redundant and meaningless speech emotion features are lessened to zero while those serviceable and informative speech emotion features are maintained and selected to the following classification step. A number of tests on Berlin database reveal that the recogni- tion rate of the SPLSR method can reach up to 79.23% and is superior to other compared dimensionality reduction methods.

Sign in / Sign up

Export Citation Format

Share Document