scholarly journals Exploiting EEG Signals and Audiovisual Feature Fusion for Video Emotion Recognition

IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 59844-59861 ◽  
Author(s):  
Baixi Xing ◽  
Hui Zhang ◽  
Kejun Zhang ◽  
Lekai Zhang ◽  
Xinda Wu ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Feature Fusion ◽  
Eeg Signals

scholarly journals Expression EEG Multimodal Emotion Recognition Method Based on the Bidirectional LSTM and Attention Mechanism

2021 ◽  
Vol 2021 ◽  
pp. 1-12
Author(s):  
Yifeng Zhao ◽  
Deyun Chen
Keyword(s):  
Emotion Recognition ◽  
Feature Fusion ◽  
Image Features ◽  
Attention Mechanism ◽  
Matlab Simulation ◽  
Eeg Signals ◽  
Recognition Method ◽  
Character Extraction ◽  
Bidirectional Lstm ◽  
Multimodal Emotion Recognition

Due to the complexity of human emotions, there are some similarities between different emotion features. The existing emotion recognition method has the problems of difficulty of character extraction and low accuracy, so the bidirectional LSTM and attention mechanism based on the expression EEG multimodal emotion recognition method are proposed. Firstly, facial expression features are extracted based on the bilinear convolution network (BCN), and EEG signals are transformed into three groups of frequency band image sequences, and BCN is used to fuse the image features to obtain the multimodal emotion features of expression EEG. Then, through the LSTM with the attention mechanism, important data is extracted in the process of timing modeling, which effectively avoids the randomness or blindness of sampling methods. Finally, a feature fusion network with a three-layer bidirectional LSTM structure is designed to fuse the expression and EEG features, which is helpful to improve the accuracy of emotion recognition. On the MAHNOB-HCI and DEAP datasets, the proposed method is tested based on the MATLAB simulation platform. Experimental results show that the attention mechanism can enhance the visual effect of the image, and compared with other methods, the proposed method can extract emotion features from expressions and EEG signals more effectively, and the accuracy of emotion recognition is higher.

scholarly journals Electroencephalogram Emotion Recognition Based on 3D Feature Fusion and Convolutional Autoencoder

2021 ◽  
Vol 15 ◽  
Author(s):  
Yanling An ◽  
Shaohai Hu ◽  
Xiaoying Duan ◽  
Ling Zhao ◽  
Caiyun Xie ◽  
...  
Keyword(s):  
Emotion Recognition ◽  
Spatial Information ◽  
Feature Fusion ◽  
Recognition Accuracy ◽  
Recognition Algorithm ◽  
Eeg Signals ◽  
Key Technologies ◽  
Convolutional Autoencoder ◽  
Valence And Arousal ◽  
Electroencephalogram Eeg

As one of the key technologies of emotion computing, emotion recognition has received great attention. Electroencephalogram (EEG) signals are spontaneous and difficult to camouflage, so they are used for emotion recognition in academic and industrial circles. In order to overcome the disadvantage that traditional machine learning based emotion recognition technology relies too much on a manual feature extraction, we propose an EEG emotion recognition algorithm based on 3D feature fusion and convolutional autoencoder (CAE). First, the differential entropy (DE) features of different frequency bands of EEG signals are fused to construct the 3D features of EEG signals, which retain the spatial information between channels. Then, the constructed 3D features are input into the CAE constructed in this paper for emotion recognition. In this paper, many experiments are carried out on the open DEAP dataset, and the recognition accuracy of valence and arousal dimensions are 89.49 and 90.76%, respectively. Therefore, the proposed method is suitable for emotion recognition tasks.

scholarly journals Affective Computing on Machine Learning-Based Emotion Recognition Using a Self-Made EEG Device

Sensors ◽  
2021 ◽  
Vol 21 (15) ◽  
pp. 5135
Author(s):  
Ngoc-Dau Mai ◽  
Boon-Giin Lee ◽  
Wan-Young Chung
Keyword(s):  
Machine Learning ◽  
Emotion Recognition ◽  
Temporal Lobe ◽  
Affective Computing ◽  
Electrode Placement ◽  
Entropy Measure ◽  
Support Vector ◽  
Emotion Classification ◽  
Eeg Signals ◽  
Computing Method

In this research, we develop an affective computing method based on machine learning for emotion recognition using a wireless protocol and a wearable electroencephalography (EEG) custom-designed device. The system collects EEG signals using an eight-electrode placement on the scalp; two of these electrodes were placed in the frontal lobe, and the other six electrodes were placed in the temporal lobe. We performed experiments on eight subjects while they watched emotive videos. Six entropy measures were employed for extracting suitable features from the EEG signals. Next, we evaluated our proposed models using three popular classifiers: a support vector machine (SVM), multi-layer perceptron (MLP), and one-dimensional convolutional neural network (1D-CNN) for emotion classification; both subject-dependent and subject-independent strategies were used. Our experiment results showed that the highest average accuracies achieved in the subject-dependent and subject-independent cases were 85.81% and 78.52%, respectively; these accuracies were achieved using a combination of the sample entropy measure and 1D-CNN. Moreover, our study investigates the T8 position (above the right ear) in the temporal lobe as the most critical channel among the proposed measurement positions for emotion classification through electrode selection. Our results prove the feasibility and efficiency of our proposed EEG-based affective computing method for emotion recognition in real-world applications.

Emotion recognition using time–frequency ridges of EEG signals based on multivariate synchrosqueezing transform

2021 ◽  
Vol 0 (0) ◽  
Author(s):  
Ahmet Mert ◽  
Hasan Huseyin Celik
Keyword(s):  
Emotion Recognition ◽  
Feature Vector ◽  
Dimensional Space ◽  
Computational Cost ◽  
Feature Space ◽  
Maximum Energy ◽  
Machine Learning Algorithms ◽  
Eeg Signals ◽  
Time Frequency ◽  
Synchrosqueezing Transform

Abstract The feasibility of using time–frequency (TF) ridges estimation is investigated on multi-channel electroencephalogram (EEG) signals for emotional recognition. Without decreasing accuracy rate of the valence/arousal recognition, the informative component extraction with low computational cost will be examined using multivariate ridge estimation. The advanced TF representation technique called multivariate synchrosqueezing transform (MSST) is used to obtain well-localized components of multi-channel EEG signals. Maximum-energy components in the 2D TF distribution are determined using TF-ridges estimation to extract instantaneous frequency and instantaneous amplitude, respectively. The statistical values of the estimated ridges are used as a feature vector to the inputs of machine learning algorithms. Thus, component information in multi-channel EEG signals can be captured and compressed into low dimensional space for emotion recognition. Mean and variance values of the five maximum-energy ridges in the MSST based TF distribution are adopted as feature vector. Properties of five TF-ridges in frequency and energy plane (e.g., mean frequency, frequency deviation, mean energy, and energy deviation over time) are computed to obtain 20-dimensional feature space. The proposed method is performed on the DEAP emotional EEG recordings for benchmarking, and the recognition rates are yielded up to 71.55, and 70.02% for high/low arousal, and high/low valence, respectively.

Sign in / Sign up

Export Citation Format

Share Document