scholarly journals Can We Ditch Feature Engineering? End-to-End Deep Learning for Affect Recognition from Physiological Sensor Data

Sensors ◽  
2020 ◽  
Vol 20 (22) ◽  
pp. 6535
Author(s):  
Maciej Dzieżyc ◽  
Martin Gjoreski ◽  
Przemysław Kazienko ◽  
Stanisław Saganowski ◽  
Matjaž Gams
Keyword(s):  
Deep Learning ◽  
Stress Test ◽  
Wearable Sensors ◽  
Physiological Signals ◽  
Trier Social Stress Test ◽  
Bayesian Optimization ◽  
Sensor Data ◽  
Affect Recognition ◽  
Affective Stimuli ◽  
End To End

To further extend the applicability of wearable sensors in various domains such as mobile health systems and the automotive industry, new methods for accurately extracting subtle physiological information from these wearable sensors are required. However, the extraction of valuable information from physiological signals is still challenging—smartphones can count steps and compute heart rate, but they cannot recognize emotions and related affective states. This study analyzes the possibility of using end-to-end multimodal deep learning (DL) methods for affect recognition. Ten end-to-end DL architectures are compared on four different datasets with diverse raw physiological signals used for affect recognition, including emotional and stress states. The DL architectures specialized for time-series classification were enhanced to simultaneously facilitate learning from multiple sensors, each having their own sampling frequency. To enable fair comparison among the different DL architectures, Bayesian optimization was used for hyperparameter tuning. The experimental results showed that the performance of the models depends on the intensity of the physiological response induced by the affective stimuli, i.e., the DL models recognize stress induced by the Trier Social Stress Test more successfully than they recognize emotional changes induced by watching affective content, e.g., funny videos. Additionally, the results showed that the CNN-based architectures might be more suitable than LSTM-based architectures for affect recognition from physiological sensors.

scholarly journals Deep ECG-Respiration Network (DeepER Net) for Recognizing Mental Stress

Sensors ◽  
2019 ◽  
Vol 19 (13) ◽  
pp. 3021 ◽  
Author(s):  
Wonju Seo ◽  
Namho Kim ◽  
Sehyeon Kim ◽  
Chanhee Lee ◽  
Sung-Min Park
Keyword(s):  
Deep Learning ◽  
Mental Stress ◽  
Detection Algorithm ◽  
Workplace Stress ◽  
Physiological Signals ◽  
Work Related ◽  
Average Accuracy ◽  
Conventional Machine ◽  
End To End

Unmanaged long-term mental stress in the workplace can lead to serious health problems and reduced productivity. To prevent this, it is important to recognize and relieve mental stress in a timely manner. Here, we propose a novel stress detection algorithm based on end-to-end deep learning using multiple physiological signals, such as electrocardiogram (ECG) and respiration (RESP) signal. To mimic workplace stress in our experiments, we used Stroop and math tasks as stressors, with each stressor being followed by a relaxation task. Herein, we recruited 18 subjects and measured both ECG and RESP signals using Zephyr BioHarness 3.0. After five-fold cross validation, the proposed network performed well, with an average accuracy of 83.9%, an average F1 score of 0.81, and an average area under the receiver operating characteristic (ROC) curve (AUC) of 0.92, demonstrating its superiority over conventional machine learning models. Furthermore, by visualizing the activation of the trained network’s neurons, we found that they were activated by specific ECG and RESP patterns. In conclusion, we successfully validated the feasibility of end-to-end deep learning using multiple physiological signals for recognition of mental stress in the workplace. We believe that this is a promising approach that will help to improve the quality of life of people suffering from long-term work-related mental stress.

scholarly journals Recognition and Repetition Counting for Complex Physical Exercises with Deep Learning

Sensors ◽  
2019 ◽  
Vol 19 (3) ◽  
pp. 714 ◽  
Author(s):  
Andrea Soro ◽  
Gino Brunner ◽  
Simon Tanner ◽  
Roger Wattenhofer
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Activity Recognition ◽  
Classification Accuracy ◽  
Sensor Data ◽  
Learning Approach ◽  
Physical Exercises ◽  
End To End

Activity recognition using off-the-shelf smartwatches is an important problem in humanactivity recognition. In this paper, we present an end-to-end deep learning approach, able to provideprobability distributions over activities from raw sensor data. We apply our methods to 10 complexfull-body exercises typical in CrossFit, and achieve a classification accuracy of 99.96%. We additionallyshow that the same neural network used for exercise recognition can also be used in repetitioncounting. To the best of our knowledge, our approach to repetition counting is novel and performswell, counting correctly within an error of 1 repetitions in 91% of the performed sets.

scholarly journals An Ensemble One Dimensional Convolutional Neural Network with Bayesian Optimization for Environmental Sound Classification

2021 ◽  
Vol 11 (10) ◽  
pp. 4660
Author(s):  
Mohammed Gamal Ragab ◽  
Said Jadid Abdulkadir ◽  
Norshakirah Aziz ◽  
Hitham Alhussian ◽  
Abubakar Bala ◽  
...  
Keyword(s):  
Neural Network ◽  
Deep Learning ◽  
Audio Signal ◽  
Bayesian Optimization ◽  
Classification Problems ◽  
Environmental Sound ◽  
One Dimensional ◽  
Sound Classification ◽  
Hyperparameter Selection ◽  
End To End

With the growth of deep learning in various classification problems, many researchers have used deep learning methods in environmental sound classification tasks. This paper introduces an end-to-end method for environmental sound classification based on a one-dimensional convolution neural network with Bayesian optimization and ensemble learning, which directly learns features representation from the audio signal. Several convolutional layers were used to capture the signal and learn various filters relevant to the classification problem. Our proposed method can deal with any audio signal length, as a sliding window divides the signal into overlapped frames. Bayesian optimization accomplished hyperparameter selection and model evaluation with cross-validation. Multiple models with different settings have been developed based on Bayesian optimization to ensure network convergence in both convex and non-convex optimization. An UrbanSound8K dataset was evaluated for the performance of the proposed end-to-end model. The experimental results achieved a classification accuracy of 94.46%, which is 5% higher than existing end-to-end approaches with fewer trainable parameters. Four measurement indices, namely: sensitivity, specificity, accuracy, precision, recall, F-measure, area under ROC curve, and the area under the precision-recall curve were used to measure the model performance. The proposed approach outperformed state-of-the-art end-to-end approaches that use hand-crafted features as input in selected measurement indices and time complexity.

scholarly journals End-To-End Deep Learning Framework for Coronavirus (COVID-19) Detection and Monitoring

Electronics ◽  
2020 ◽  
Vol 9 (9) ◽  
pp. 1439 ◽  
Author(s):  
Nora El-Rashidy ◽  
Shaker El-Sappagh ◽  
S. M. Riazul Islam ◽  
Hazem M. El-Bakry ◽  
Samir Abdelrazek
Keyword(s):  
Deep Learning ◽  
Network Architecture ◽  
Fog Computing ◽  
Wearable Sensors ◽  
Wireless Body Area Network ◽  
Healthcare Systems ◽  
Mobile App ◽  
Cloud Layer ◽  
Area Network ◽  
End To End

Coronavirus (COVID-19) is a new virus of viral pneumonia. It can outbreak in the world through person-to-person transmission. Although several medical companies provide cooperative monitoring healthcare systems, these solutions lack offering of the end-to-end management of the disease. The main objective of the proposed framework is to bridge the current gap between current technologies and healthcare systems. The wireless body area network, cloud computing, fog computing, and clinical decision support system are integrated to provide a comprehensive and complete model for disease detection and monitoring. By monitoring a person with COVID-19 in real time, physicians can guide patients with the right decisions. The proposed framework has three main layers (i.e., a patient layer, cloud layer, and hospital layer). In the patient layer, the patient is tracked through a set of wearable sensors and a mobile app. In the cloud layer, a fog network architecture is proposed to solve the issues of storage and data transmission. In the hospital layer, we propose a convolutional neural network-based deep learning model for COVID-19 detection based on patient’s X-ray scan images and transfer learning. The proposed model achieved promising results compared to the state-of-the art (i.e., accuracy of 97.95% and specificity of 98.85%). Our framework is a useful application, through which we expect significant effects on COVID-19 proliferation and considerable lowering in healthcare expenses.

Messung der Stimmfrequenz im Paargespräch – Chancen für Diagnostik und Intervention in der Paartherapie

2020 ◽  
Vol 68 (4) ◽  
pp. 217-227 ◽  
Author(s):  
Lisanne J. Bulling ◽  
Isabella C. Bertschi ◽  
Céline C. Stadelmann ◽  
Tina Niederer ◽  
Guy Bodenmann
Keyword(s):  
Social Stress ◽  
Stress Test ◽  
Trier Social Stress Test

Zusammenfassung. Die vorliegende Arbeit stellt die bisherigen empirischen Befunde zur Sprachgrundfrequenz (f0) in Paargesprächen vor und untersucht, wie sich die f0 nach einer experimentellen Stressinduktion im anschließenden spontanen Gespräch zwischen den Partner_innen verändert, wie die f0 mit der verbalen Stressäußerung zusammenhängt und wie sie zwischen den beiden Partner_innen kovariiert. Von 128 heterosexuellen Paaren nahm jeweils eine Person pro Paar am Trier Social Stress Test (TSST) teil. Die dem TSST vorangehende und anschließende naturalistische Interaktion zwischen den Partner_innen wurde gefilmt und nach Gesprächsthema und Art der Stressäußerung kodiert. Wie vorherige Studien zur f0 im Paargespräch zeigte auch die vorliegende Studie, dass die f0 wichtige Informationen über die Partnerschaft enthält. Während eine Erhöhung der f0 in Gesprächen über einen paarinternen Stressor (d.h. bei Konfliktgesprächen) mit negativen Kommunikationsmustern einherging, zeigte die vorliegende Studie, dass die f0 bei Gesprächen über einen paarexternen Stressor (d.h. beim TSST) mit emotionsorientierten Stressäußerungen einherging, also einer für den Stressbewältigungsprozess förderlichen Art der Kommunikation. Die Oszillatorenmodelle zeigen darüber hinaus, dass eine Kopplung der f0 zwischen den Partner_innen besteht, was darauf hindeutet, dass die nicht gestressten Partner_innen auf die paraverbalen Stressäußerungen der gestressten Partner_innen mit ihren eigenen paraverbalen Stressäußerungen reagieren.

Human Activity Recognition using Fourier Transform Inspired Deep Learning Combination Model

2019 ◽  
Vol 9 (1) ◽  
pp. 16-31
Author(s):  
Kyungkoo Jun
Keyword(s):  
Fourier Transform ◽  
Deep Learning ◽  
Short Term Memory ◽  
Window Size ◽  
Sensor Data ◽  
Data Sets ◽  
Data Set ◽  
Proposed Model ◽  
Testing Data ◽  
Labeling Scheme

Background & Objective: This paper proposes a Fourier transform inspired method to classify human activities from time series sensor data. Methods: Our method begins by decomposing 1D input signal into 2D patterns, which is motivated by the Fourier conversion. The decomposition is helped by Long Short-Term Memory (LSTM) which captures the temporal dependency from the signal and then produces encoded sequences. The sequences, once arranged into the 2D array, can represent the fingerprints of the signals. The benefit of such transformation is that we can exploit the recent advances of the deep learning models for the image classification such as Convolutional Neural Network (CNN). Results: The proposed model, as a result, is the combination of LSTM and CNN. We evaluate the model over two data sets. For the first data set, which is more standardized than the other, our model outperforms previous works or at least equal. In the case of the second data set, we devise the schemes to generate training and testing data by changing the parameters of the window size, the sliding size, and the labeling scheme. Conclusion: The evaluation results show that the accuracy is over 95% for some cases. We also analyze the effect of the parameters on the performance.

An intervention study on the effectiveness of an anti-stress-app on psychophysiological acute and chronic stress (Preprint)

2018 ◽  
Author(s):  
Franziska Lautenbach
Keyword(s):  
Chronic Stress ◽  
Acute Stress ◽  
Physiological Stress ◽  
Stress Test ◽  
Experimental Studies ◽  
Trier Social Stress Test ◽  
Control Group ◽  
Subjective Perception ◽  
Face To Face ◽  
Acute And Chronic Stress

BACKGROUND Dealing with stress is of central importance. Lately, smartphone applications (apps) are deployed in stress interventions as they offer maximal flexibility for users. First results of experimental studies show that anti-stress apps effect subjective perception of stress positively (Ly et al., 2014). However, current literature lacks studies on physiological stress reactions (e.g., cortisol), although they are of special interest to health issues. OBJECTIVE Therefore, the aim of this study was to investigate the effectiveness of an anti-stress app in chronic and acute stress reduction on a physiological (cortisol) and psychological level (subjective perception of stress) in comparison to a face-to-face and a control group in a pre-post design, for the first time. METHODS Sixty-two participants took part in the pretesting procedure (drop-out of 53 %). Based on age, gender, physical activity and subjectively perceived acute stress due to the Trier Social Stress Test for groups (TSST-G; von Dawans et al., 2011) as well as based on subjectively chronic stress assessed during the pretest, participants were parallelized in three groups (anti-stress-app: n = 10, face-to-face: n = 11, control group: n = 9). RESULTS After six weeks of the cognitive-based resource-oriented intervention, participants were exposed to the TSST-G for post testing. Results did not show a change of cortisol secretion or cognitive appraisal of the acute stressor. Further, no changes were detected in the chronic physiological stress reaction. CONCLUSIONS Possible causes are discussed extensively. CLINICALTRIAL no

Sign in / Sign up

Export Citation Format

Share Document