scholarly journals Multi-Sensor Fusion for Activity Recognition—A Survey

Sensors ◽  
2019 ◽  
Vol 19 (17) ◽  
pp. 3808 ◽  
Author(s):  
Antonio A. Aguileta ◽  
Ramon F. Brena ◽  
Oscar Mayora ◽  
Erik Molino-Minero-Re ◽  
Luis A. Trejo
Keyword(s):  
Sensor Fusion ◽  
Activity Recognition ◽  
Sports Medicine ◽  
Good Practice ◽  
Sensor Data ◽  
Personal Safety ◽  
Multiple Sensors ◽  
Concurrent Use ◽  
Fusion Methods ◽  
The Many

In Ambient Intelligence (AmI), the activity a user is engaged in is an essential part of the context, so its recognition is of paramount importance for applications in areas like sports, medicine, personal safety, and so forth. The concurrent use of multiple sensors for recognition of human activities in AmI is a good practice because the information missed by one sensor can sometimes be provided by the others and many works have shown an accuracy improvement compared to single sensors. However, there are many different ways of integrating the information of each sensor and almost every author reporting sensor fusion for activity recognition uses a different variant or combination of fusion methods, so the need for clear guidelines and generalizations in sensor data integration seems evident. In this survey we review, following a classification, the many fusion methods for information acquired from sensors that have been proposed in the literature for activity recognition; we examine their relative merits, either as they are reported and sometimes even replicated and a comparison of these methods is made, as well as an assessment of the trends in the area.

scholarly journals Gas Detection and Identification Using Multimodal Artificial Intelligence Based Sensor Fusion

2021 ◽  
Vol 4 (1) ◽  
pp. 3
Author(s):  
Parag Narkhede ◽  
Rahee Walambe ◽  
Shruti Mandaokar ◽  
Pulkit Chandel ◽  
Ketan Kotecha ◽  
...  
Keyword(s):  
Sensor Fusion ◽  
Network Architecture ◽  
Cost Effective ◽  
Sensor Data ◽  
Dense Layer ◽  
Gaseous Emissions ◽  
Multiple Sensors ◽  
Single Output ◽  
Novel Approach ◽  
Single Sensor

With the rapid industrialization and technological advancements, innovative engineering technologies which are cost effective, faster and easier to implement are essential. One such area of concern is the rising number of accidents happening due to gas leaks at coal mines, chemical industries, home appliances etc. In this paper we propose a novel approach to detect and identify the gaseous emissions using the multimodal AI fusion techniques. Most of the gases and their fumes are colorless, odorless, and tasteless, thereby challenging our normal human senses. Sensing based on a single sensor may not be accurate, and sensor fusion is essential for robust and reliable detection in several real-world applications. We manually collected 6400 gas samples (1600 samples per class for four classes) using two specific sensors: the 7-semiconductor gas sensors array, and a thermal camera. The early fusion method of multimodal AI, is applied The network architecture consists of a feature extraction module for individual modality, which is then fused using a merged layer followed by a dense layer, which provides a single output for identifying the gas. We obtained the testing accuracy of 96% (for fused model) as opposed to individual model accuracies of 82% (based on Gas Sensor data using LSTM) and 93% (based on thermal images data using CNN model). Results demonstrate that the fusion of multiple sensors and modalities outperforms the outcome of a single sensor.

scholarly journals Sensor Fusion for Recognition of Activities of Daily Living

Sensors ◽  
2018 ◽  
Vol 18 (11) ◽  
pp. 4029 ◽  
Author(s):  
Jiaxuan Wu ◽  
Yunfei Feng ◽  
Peng Sun
Keyword(s):  
Time Series ◽  
Sensor Fusion ◽  
Daily Living ◽  
Single Point ◽  
Recognition System ◽  
Sensor Data ◽  
Multiple Sensors ◽  
Life Patterns ◽  
Functional Capabilities ◽  
Correction Technique

Activity of daily living (ADL) is a significant predictor of the independence and functional capabilities of an individual. Measurements of ADLs help to indicate one’s health status and capabilities of quality living. Recently, the most common ways to capture ADL data are far from automation, including a costly 24/7 observation by a designated caregiver, self-reporting by the user laboriously, or filling out a written ADL survey. Fortunately, ubiquitous sensors exist in our surroundings and on electronic devices in the Internet of Things (IoT) era. We proposed the ADL Recognition System that utilizes the sensor data from a single point of contact, such as smartphones, and conducts time-series sensor fusion processing. Raw data is collected from the ADL Recorder App constantly running on a user’s smartphone with multiple embedded sensors, including the microphone, Wi-Fi scan module, heading orientation of the device, light proximity, step detector, accelerometer, gyroscope, magnetometer, etc. Key technologies in this research cover audio processing, Wi-Fi indoor positioning, proximity sensing localization, and time-series sensor data fusion. By merging the information of multiple sensors, with a time-series error correction technique, the ADL Recognition System is able to accurately profile a person’s ADLs and discover his life patterns. This paper is particularly concerned with the care for the older adults who live independently.

scholarly journals Sensor Data Acquisition and Multimodal Sensor Fusion for Human Activity Recognition Using Deep Learning

Sensors ◽  
2019 ◽  
Vol 19 (7) ◽  
pp. 1716 ◽  
Author(s):  
Seungeun Chung ◽  
Jiyoun Lim ◽  
Kyoung Ju Noh ◽  
Gague Kim ◽  
Hyuntae Jeong
Keyword(s):  
Deep Learning ◽  
Data Acquisition ◽  
Sensor Fusion ◽  
Activity Recognition ◽  
Human Activity ◽  
Short Term Memory ◽  
Sampling Rate ◽  
Human Activity Recognition ◽  
Sensor Data ◽  
Activity Data

In this paper, we perform a systematic study about the on-body sensor positioning and data acquisition details for Human Activity Recognition (HAR) systems. We build a testbed that consists of eight body-worn Inertial Measurement Units (IMU) sensors and an Android mobile device for activity data collection. We develop a Long Short-Term Memory (LSTM) network framework to support training of a deep learning model on human activity data, which is acquired in both real-world and controlled environments. From the experiment results, we identify that activity data with sampling rate as low as 10 Hz from four sensors at both sides of wrists, right ankle, and waist is sufficient in recognizing Activities of Daily Living (ADLs) including eating and driving activity. We adopt a two-level ensemble model to combine class-probabilities of multiple sensor modalities, and demonstrate that a classifier-level sensor fusion technique can improve the classification performance. By analyzing the accuracy of each sensor on different types of activity, we elaborate custom weights for multimodal sensor fusion that reflect the characteristic of individual activities.

Sensor Fusion Algorithm that Has Resilience under Autonomous Driving Conditions: A Survey

2017 ◽  
Vol 865 ◽  
pp. 429-433
Author(s):  
Sung Bum Park ◽  
Hyeok Chan Kwon ◽  
Dong Hoon Lee
Keyword(s):  
Sensor Fusion ◽  
Autonomous Driving ◽  
Sensor Data ◽  
Fusion Algorithm ◽  
Multiple Sensors ◽  
Autonomous Cars ◽  
Driving Conditions

Autonomous cars recognize the surroundings through multiple sensors and make decisions to control the car in order to arrive at destination without driver's interventions. In such environment, if sensor data forgery occurs, it could lead to a big (critical) accident that could threaten the life of the driver. In the paper, a research on a way to get accurate driving information through sensor fusion algorithm that has resilience against data forgery and modulation will be discussed.

A NEW APPROACH FOR MULTI-SYSTEM/SENSOR DECISION FUSION BASED ON JOINT MEASURES

2004 ◽  
Vol 01 (02) ◽  
pp. 109-120 ◽  
Author(s):  
ALI J. RASHIDI ◽  
M. HASSAN GHASSEMIAN
Keyword(s):  
High Performance ◽  
Decision Fusion ◽  
Sensor Data ◽  
Commission Errors ◽  
Multiple Sensors ◽  
Spectral Bands ◽  
Mathematical Properties ◽  
Fusion Methods ◽  
Local Results ◽  
Fusion Scheme

This article describes the joint measures method as a new powerful method for the development of a high performance multi-sensor data/image fusion scheme at the decision level. The images are received from distributed multiple sensors, which sense the targets in different spectral bands including visible, infrared, thermal and microwave. At first, we study the decision fusion methods, including voting schemes, rank based algorithm, Bayesian inference, and the Dempster-Shafer method. Then, we extract the mathematical properties of multi-sensor local classification results and use them for modeling of the classifier performances by the two new measures, i.e. the plausibility and correctness. Then we establish the plausibility and correctness distribution vectors and matrices for introducing the two improvements of the Dempster-Shafer method, i.e. the DS (CM) and DS (PM) methods. After that we introduce the joint measures decision fusion method based on using these two measures jointly. The Joint Measures Method (JMM) can deal with any decision fusion problem in the case of uncertain local classifiers results as well as clear local classifiers results. Finally, we deploy the new and previous methods for the fusion of the two different sets of multispectral image classification local results and we also compare their reliabilities, the commission errors and the omission errors. The results obviously show that the DS (PM), DS (CM) and JMM methods which use the special properties of the local classifiers and classes, have much better accuracies and reliabilities than other methods. In addition, we show that the reliability of the JMM is at least 3% higher than all other methods.

scholarly journals Degeneration-Aware Localization with Arbitrary Global-Local Sensor Fusion

Sensors ◽  
2021 ◽  
Vol 21 (12) ◽  
pp. 4042
Author(s):  
Xiaqing Ding ◽  
Fuzhang Han ◽  
Tong Yang ◽  
Yue Wang ◽  
Rong Xiong
Keyword(s):  
Sensor Fusion ◽  
Single Type ◽  
Fusion Method ◽  
Global Localization ◽  
Related System ◽  
Multiple Sensors ◽  
Fusion Methods ◽  
Local Reference ◽  
Real World Datasets ◽  
Global And Local

Global localization is a fundamental ability for mobile robots. Considering the limitation of single type of sensor, fusing measurements from multiple sensors with complementary properties is a valuable task for study. In this paper, we propose a decoupled optimization-based framework for global–local sensor fusion, which fuses the intermittent 3D global positions and high-frequent 6D odometry poses to infer the 6D global localization results in real-time. The fusion process is formulated as estimating the relative transformation between global and local reference coordinates, translational extrinsic calibration, and the scale of the local pose estimator. We validate the full observability of the system under general movements, and further analyze the degenerated movement patterns where some related system state would be unobservable. A degeneration-aware sensor fusion method is designed which detects the degenerated directions before optimization, and adds constraints specifically along these directions to relieve the effect of the noise. The proposed degeneration-aware global–local sensor fusion method is validated in both simulation and real-world datasets with different sensor configurations, and shows its effectiveness in terms of accuracy and robustness compared with other decoupled sensor fusion methods for global localization.

scholarly journals Centralised and Decentralised Sensor Fusion-Based Emergency Brake Assist

Sensors ◽  
2021 ◽  
Vol 21 (16) ◽  
pp. 5422
Author(s):  
Ankur Deo ◽  
Vasile Palade ◽  
Md. Nazmul Huda
Keyword(s):  
Sensor Fusion ◽  
Computational Cost ◽  
Environmental Parameters ◽  
Frame Rate ◽  
Driver Assistance ◽  
Driver Assistance Systems ◽  
Fusion Algorithm ◽  
Multiple Sensors ◽  
Fusion Methods ◽  
Brake Assist

Many advanced driver assistance systems (ADAS) are currently trying to utilise multi-sensor architectures, where the driver assistance algorithm receives data from a multitude of sensors. As mono-sensor systems cannot provide reliable and consistent readings under all circumstances because of errors and other limitations, fusing data from multiple sensors ensures that the environmental parameters are perceived correctly and reliably for most scenarios, thereby substantially improving the reliability of the multi-sensor-based automotive systems. This paper first highlights the significance of efficiently fusing data from multiple sensors in ADAS features. An emergency brake assist (EBA) system is showcased using multiple sensors, namely, a light detection and ranging (LiDAR) sensor and camera. The architectures of the proposed ‘centralised’ and ‘decentralised’ sensor fusion approaches for EBA are discussed along with their constituents, i.e., the detection algorithms, the fusion algorithm, and the tracking algorithm. The centralised and decentralised architectures are built and analytically compared, and the performance of these two fusion architectures for EBA are evaluated in terms of speed of execution, accuracy, and computational cost. While both fusion methods are seen to drive the EBA application at an acceptable frame rate (~20fps or higher) on an Intel i5-based Ubuntu system, it was concluded through the experiments and analytical comparisons that the decentralised fusion-driven EBA leads to higher accuracy; however, it has the downside of a higher computational cost. The centralised fusion-driven EBA yields comparatively less accurate results, but with the benefits of a higher frame rate and lesser computational cost.

scholarly journals Deep Convolutional Neural Network with RNNs for Complex Activity Recognition Using Wrist-Worn Wearable Sensor Data

Electronics ◽  
2021 ◽  
Vol 10 (14) ◽  
pp. 1685
Author(s):  
Sakorn Mekruksavanich ◽  
Anuchit Jitpattanakul
Keyword(s):  
Neural Networks ◽  
Activity Recognition ◽  
Human Activities ◽  
Recognition Performance ◽  
Confusion Matrix ◽  
Experimental Studies ◽  
Industrial Applications ◽  
Sensor Data ◽  
Complex Activity ◽  
Activity Data

Sensor-based human activity recognition (S-HAR) has become an important and high-impact topic of research within human-centered computing. In the last decade, successful applications of S-HAR have been presented through fruitful academic research and industrial applications, including for healthcare monitoring, smart home controlling, and daily sport tracking. However, the growing requirements of many current applications for recognizing complex human activities (CHA) have begun to attract the attention of the HAR research field when compared with simple human activities (SHA). S-HAR has shown that deep learning (DL), a type of machine learning based on complicated artificial neural networks, has a significant degree of recognition efficiency. Convolutional neural networks (CNNs) and recurrent neural networks (RNNs) are two different types of DL methods that have been successfully applied to the S-HAR challenge in recent years. In this paper, we focused on four RNN-based DL models (LSTMs, BiLSTMs, GRUs, and BiGRUs) that performed complex activity recognition tasks. The efficiency of four hybrid DL models that combine convolutional layers with the efficient RNN-based models was also studied. Experimental studies on the UTwente dataset demonstrated that the suggested hybrid RNN-based models achieved a high level of recognition performance along with a variety of performance indicators, including accuracy, F1-score, and confusion matrix. The experimental results show that the hybrid DL model called CNN-BiGRU outperformed the other DL models with a high accuracy of 98.89% when using only complex activity data. Moreover, the CNN-BiGRU model also achieved the highest recognition performance in other scenarios (99.44% by using only simple activity data and 98.78% with a combination of simple and complex activities).

Sign in / Sign up

Export Citation Format

Share Document