scholarly journals SemImput: Bridging Semantic Imputation with Deep Learning for Complex Human Activity Recognition

Sensors ◽  
2020 ◽  
Vol 20 (10) ◽  
pp. 2771
Author(s):  
Muhammad Asif Razzaq ◽  
Ian Cleland ◽  
Chris Nugent ◽  
Sungyoung Lee
Keyword(s):  
Deep Learning ◽  
Research Area ◽  
Data Sources ◽  
Sensor Data ◽  
Similarity Learning ◽  
Multiple Sensors ◽  
Important Research Area ◽  
Artificial Neural Network Ann ◽  
Public Datasets ◽  
Syntactic Differences

The recognition of activities of daily living (ADL) in smart environments is a well-known and an important research area, which presents the real-time state of humans in pervasive computing. The process of recognizing human activities generally involves deploying a set of obtrusive and unobtrusive sensors, pre-processing the raw data, and building classification models using machine learning (ML) algorithms. Integrating data from multiple sensors is a challenging task due to dynamic nature of data sources. This is further complicated due to semantic and syntactic differences in these data sources. These differences become even more complex if the data generated is imperfect, which ultimately has a direct impact on its usefulness in yielding an accurate classifier. In this study, we propose a semantic imputation framework to improve the quality of sensor data using ontology-based semantic similarity learning. This is achieved by identifying semantic correlations among sensor events through SPARQL queries, and by performing a time-series longitudinal imputation. Furthermore, we applied deep learning (DL) based artificial neural network (ANN) on public datasets to demonstrate the applicability and validity of the proposed approach. The results showed a higher accuracy with semantically imputed datasets using ANN. We also presented a detailed comparative analysis, comparing the results with the state-of-the-art from the literature. We found that our semantic imputed datasets improved the classification accuracy with 95.78% as a higher one thus proving the effectiveness and robustness of learned models.

scholarly journals Deep Learning based Human Action Recognition

2021 ◽  
Vol 40 ◽  
pp. 03014
Author(s):  
Ritik Pandey ◽  
Yadnesh Chikhale ◽  
Ritik Verma ◽  
Deepali Patil
Keyword(s):  
Deep Learning ◽  
Image Classification ◽  
Action Recognition ◽  
Human Action Recognition ◽  
Human Action ◽  
Research Area ◽  
Video Clips ◽  
Object Interaction ◽  
Important Research Area ◽  
Multiple Frames

Human action recognition has become an important research area in the fields of computer vision, image processing, and human-machine or human-object interaction due to its large number of real time applications. Action recognition is the identification of different actions from video clips (an arrangement of 2D frames) where the action may be performed in the video. This is a general construction of image classification tasks to multiple frames and then collecting the predictions from each frame. Different approaches are proposed in literature to improve the accuracy in recognition. In this paper we proposed a deep learning based model for Recognition and the main focus is on the CNN model for image classification. The action videos are converted into frames and pre-processed before sending to our model for recognizing different actions accurately..

scholarly journals Violence Detection With Two-Stream Neural Network Based on C3D

2021 ◽  
Vol 15 (4) ◽  
pp. 1-17
Author(s):  
zanzan Lu ◽  
Xuewen Xia ◽  
Hongrun Wu ◽  
Chen Yang
Keyword(s):  
Large Scale ◽  
Research Area ◽  
Video Data ◽  
Small Scale ◽  
Stream Network ◽  
Generalization Ability ◽  
Violence Detection ◽  
Good Ability ◽  
Important Research Area ◽  
Public Datasets

In recent years, violence detection has gradually turned into an important research area in computer vision, and have proposed many models with high accuracy. However, the unsatisfactory generalization ability of these methods over different datasets. In this paper, the authors propose a violence detection method based on C3D two-stream network for spatiotemporal features. Firstly, the authors preprocess the video data of RGB stream and optical stream respectively. Secondly, the authors feed the data into two C3D networks to extract features from the RGB flow and the optical flow respectively. Third, the authors fuse the features extracted by the two networks to obtain a final prediction result. To testify the performance of the proposed model, four different datasets (two public datasets and two self-built datasets) are selected in this paper. The experimental results show that our model has good generalization ability compared to state-of-the-art methods, since it not only has good ability on large-scale datasets, but also performs well on small-scale datasets.

scholarly journals THE SEN1-2 DATASET FOR DEEP LEARNING IN SAR-OPTICAL DATA FUSION

2018 ◽  
Vol IV-1 ◽  
pp. 141-146 ◽  
Author(s):  
M. Schmitt ◽  
L. H. Hughes ◽  
X. X. Zhu
Keyword(s):  
Remote Sensing ◽  
Deep Learning ◽  
Data Fusion ◽  
Training Data ◽  
Sensor Data ◽  
Optical Data ◽  
Multiple Sensors ◽  
Optical Images ◽  
Multi Sensor Data Fusion ◽  
Learning Techniques

<p><strong>Abstract.</strong> While deep learning techniques have an increasing impact on many technical fields, gathering sufficient amounts of training data is a challenging problem in remote sensing. In particular, this holds for applications involving data from multiple sensors with heterogeneous characteristics. One example for that is the fusion of synthetic aperture radar (SAR) data and optical imagery. With this paper, we publish the <i>SEN1-2</i> dataset to foster deep learning research in SAR-optical data fusion. <i>SEN1-2</i> comprises 282;384 pairs of corresponding image patches, collected from across the globe and throughout all meteorological seasons. Besides a detailed description of the dataset, we show exemplary results for several possible applications, such as SAR image colorization, SAR-optical image matching, and creation of artificial optical images from SAR input data. Since <i>SEN1-2</i> is the first large open dataset of this kind, we believe it will support further developments in the field of deep learning for remote sensing as well as multi-sensor data fusion.</p>

Deep Learning Based High-Resolution Remote Sensing Image classification

2017 ◽  
Vol 7 (10) ◽  
pp. 22
Author(s):  
Sumit Kaur
Keyword(s):  
Machine Learning ◽  
Remote Sensing ◽  
Deep Learning ◽  
Image Classification ◽  
Language Processing ◽  
Object Perception ◽  
Remote Sensing Image ◽  
Research Area ◽  
Remote Sensing Image Classification ◽  
Unsupervised Algorithms

Abstract- Deep learning is an emerging research area in machine learning and pattern recognition field which has been presented with the goal of drawing Machine Learning nearer to one of its unique objectives, Artificial Intelligence. It tries to mimic the human brain, which is capable of processing and learning from the complex input data and solving different kinds of complicated tasks well. Deep learning (DL) basically based on a set of supervised and unsupervised algorithms that attempt to model higher level abstractions in data and make it self-learning for hierarchical representation for classification. In the recent years, it has attracted much attention due to its state-of-the-art performance in diverse areas like object perception, speech recognition, computer vision, collaborative filtering and natural language processing. This paper will present a survey on different deep learning techniques for remote sensing image classification. 

Human Activity Recognition using Fourier Transform Inspired Deep Learning Combination Model

2019 ◽  
Vol 9 (1) ◽  
pp. 16-31
Author(s):  
Kyungkoo Jun
Keyword(s):  
Fourier Transform ◽  
Deep Learning ◽  
Short Term Memory ◽  
Window Size ◽  
Sensor Data ◽  
Data Sets ◽  
Data Set ◽  
Proposed Model ◽  
Testing Data ◽  
Labeling Scheme

Background & Objective: This paper proposes a Fourier transform inspired method to classify human activities from time series sensor data. Methods: Our method begins by decomposing 1D input signal into 2D patterns, which is motivated by the Fourier conversion. The decomposition is helped by Long Short-Term Memory (LSTM) which captures the temporal dependency from the signal and then produces encoded sequences. The sequences, once arranged into the 2D array, can represent the fingerprints of the signals. The benefit of such transformation is that we can exploit the recent advances of the deep learning models for the image classification such as Convolutional Neural Network (CNN). Results: The proposed model, as a result, is the combination of LSTM and CNN. We evaluate the model over two data sets. For the first data set, which is more standardized than the other, our model outperforms previous works or at least equal. In the case of the second data set, we devise the schemes to generate training and testing data by changing the parameters of the window size, the sliding size, and the labeling scheme. Conclusion: The evaluation results show that the accuracy is over 95% for some cases. We also analyze the effect of the parameters on the performance.

scholarly journals Evaluation of the feasibility of explainable computer-aided detection of cardiomegaly on chest radiographs using deep learning

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Mu Sook Lee ◽  
Yong Soo Kim ◽  
Minki Kim ◽  
Muhammad Usman ◽  
Shi Sub Byon ◽  
...  
Keyword(s):  
Deep Learning ◽  
Diagnostic Performance ◽  
Absolute Error ◽  
Training Dataset ◽  
Computer Aided Detection ◽  
Test Dataset ◽  
Cardiothoracic Ratio ◽  
Computer Aided ◽  
Chest X Ray ◽  
Public Datasets

AbstractWe examined the feasibility of explainable computer-aided detection of cardiomegaly in routine clinical practice using segmentation-based methods. Overall, 793 retrospectively acquired posterior–anterior (PA) chest X-ray images (CXRs) of 793 patients were used to train deep learning (DL) models for lung and heart segmentation. The training dataset included PA CXRs from two public datasets and in-house PA CXRs. Two fully automated segmentation-based methods using state-of-the-art DL models for lung and heart segmentation were developed. The diagnostic performance was assessed and the reliability of the automatic cardiothoracic ratio (CTR) calculation was determined using the mean absolute error and paired t-test. The effects of thoracic pathological conditions on performance were assessed using subgroup analysis. One thousand PA CXRs of 1000 patients (480 men, 520 women; mean age 63 ± 23 years) were included. The CTR values derived from the DL models and diagnostic performance exhibited excellent agreement with reference standards for the whole test dataset. Performance of segmentation-based methods differed based on thoracic conditions. When tested using CXRs with lesions obscuring heart borders, the performance was lower than that for other thoracic pathological findings. Thus, segmentation-based methods using DL could detect cardiomegaly; however, the feasibility of computer-aided detection of cardiomegaly without human intervention was limited.

scholarly journals Similarity Embedding Networks for Robust Human Activity Recognition

2021 ◽  
Vol 15 (6) ◽  
pp. 1-17
Author(s):  
Chenglin Li ◽  
Carrie Lu Tong ◽  
Di Niu ◽  
Bei Jiang ◽  
Xiao Zuo ◽  
...  
Keyword(s):  
Activity Recognition ◽  
Human Activity ◽  
Short Term Memory ◽  
Real Space ◽  
Human Activity Recognition ◽  
Sensor Data ◽  
Activity Data ◽  
Extensive Evaluation ◽  
Sensor Signals ◽  
Public Datasets

Deep learning models for human activity recognition (HAR) based on sensor data have been heavily studied recently. However, the generalization ability of deep models on complex real-world HAR data is limited by the availability of high-quality labeled activity data, which are hard to obtain. In this article, we design a similarity embedding neural network that maps input sensor signals onto real vectors through carefully designed convolutional and Long Short-Term Memory (LSTM) layers. The embedding network is trained with a pairwise similarity loss, encouraging the clustering of samples from the same class in the embedded real space, and can be effectively trained on a small dataset and even on a noisy dataset with mislabeled samples. Based on the learned embeddings, we further propose both nonparametric and parametric approaches for activity recognition. Extensive evaluation based on two public datasets has shown that the proposed similarity embedding network significantly outperforms state-of-the-art deep models on HAR classification tasks, is robust to mislabeled samples in the training set, and can also be used to effectively denoise a noisy dataset.

scholarly journals Representation Learning for Fine-Grained Change Detection

Sensors ◽  
2021 ◽  
Vol 21 (13) ◽  
pp. 4486
Author(s):  
Niall O’Mahony ◽  
Sean Campbell ◽  
Lenka Krpalkova ◽  
Anderson Carvalho ◽  
Joseph Walsh ◽  
...  
Keyword(s):  
Deep Learning ◽  
Change Detection ◽  
Model Calibration ◽  
State Of The Art ◽  
Representation Learning ◽  
Machine Intelligence ◽  
The State ◽  
Sensor Data ◽  
Fine Grained ◽  
Learning Techniques

Fine-grained change detection in sensor data is very challenging for artificial intelligence though it is critically important in practice. It is the process of identifying differences in the state of an object or phenomenon where the differences are class-specific and are difficult to generalise. As a result, many recent technologies that leverage big data and deep learning struggle with this task. This review focuses on the state-of-the-art methods, applications, and challenges of representation learning for fine-grained change detection. Our research focuses on methods of harnessing the latent metric space of representation learning techniques as an interim output for hybrid human-machine intelligence. We review methods for transforming and projecting embedding space such that significant changes can be communicated more effectively and a more comprehensive interpretation of underlying relationships in sensor data is facilitated. We conduct this research in our work towards developing a method for aligning the axes of latent embedding space with meaningful real-world metrics so that the reasoning behind the detection of change in relation to past observations may be revealed and adjusted. This is an important topic in many fields concerned with producing more meaningful and explainable outputs from deep learning and also for providing means for knowledge injection and model calibration in order to maintain user confidence.

scholarly journals Examining Deep Learning Models with Multiple Data Sources for COVID-19 Forecasting

2020 ◽  
Author(s):  
Lijing Wang ◽  
Aniruddha Adiga ◽  
Srinivasan Venkatramanan ◽  
Jiangzhuo Chen ◽  
Bryan Lewis ◽  
...  
Keyword(s):  
Deep Learning ◽  
Data Sources ◽  
Learning Models ◽  
Multiple Data Sources ◽  
Multiple Data

scholarly journals Interpretable deep learning for the remote characterisation of ambulation in multiple sclerosis using smartphones

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Andrew P. Creagh ◽  
Florian Lipsmeier ◽  
Michael Lindemann ◽  
Maarten De Vos
Keyword(s):  
Multiple Sclerosis ◽  
Deep Learning ◽  
Inertial Sensor ◽  
Heterogeneous Data ◽  
Fine Tuning ◽  
Sensor Data ◽  
Support Vector ◽  
Deep Convolutional Neural Networks ◽  
Healthcare Applications ◽  
Feature Based

AbstractThe emergence of digital technologies such as smartphones in healthcare applications have demonstrated the possibility of developing rich, continuous, and objective measures of multiple sclerosis (MS) disability that can be administered remotely and out-of-clinic. Deep Convolutional Neural Networks (DCNN) may capture a richer representation of healthy and MS-related ambulatory characteristics from the raw smartphone-based inertial sensor data than standard feature-based methodologies. To overcome the typical limitations associated with remotely generated health data, such as low subject numbers, sparsity, and heterogeneous data, a transfer learning (TL) model from similar large open-source datasets was proposed. Our TL framework leveraged the ambulatory information learned on human activity recognition (HAR) tasks collected from wearable smartphone sensor data. It was demonstrated that fine-tuning TL DCNN HAR models towards MS disease recognition tasks outperformed previous Support Vector Machine (SVM) feature-based methods, as well as DCNN models trained end-to-end, by upwards of 8–15%. A lack of transparency of “black-box” deep networks remains one of the largest stumbling blocks to the wider acceptance of deep learning for clinical applications. Ensuing work therefore aimed to visualise DCNN decisions attributed by relevance heatmaps using Layer-Wise Relevance Propagation (LRP). Through the LRP framework, the patterns captured from smartphone-based inertial sensor data that were reflective of those who are healthy versus people with MS (PwMS) could begin to be established and understood. Interpretations suggested that cadence-based measures, gait speed, and ambulation-related signal perturbations were distinct characteristics that distinguished MS disability from healthy participants. Robust and interpretable outcomes, generated from high-frequency out-of-clinic assessments, could greatly augment the current in-clinic assessment picture for PwMS, to inform better disease management techniques, and enable the development of better therapeutic interventions.

Sign in / Sign up

Export Citation Format

Share Document