scholarly journals Exploring Semi-Supervised Methods for Labeling Support in Multimodal Datasets

Sensors ◽  
2018 ◽  
Vol 18 (8) ◽  
pp. 2639 ◽  
Author(s):  
Alexander Diete ◽  
Timo Sztyler ◽  
Heiner Stuckenschmidt
Keyword(s):  
Template Matching ◽  
Daily Living ◽  
Inertial Sensor ◽  
Video Recording ◽  
Sensor Data ◽  
Time Intervals ◽  
Video Recordings ◽  
Acceleration Data ◽  
Small Set ◽  
Supervised Methods

Working with multimodal datasets is a challenging task as it requires annotations which often are time consuming and difficult to acquire. This includes in particular video recordings which often need to be watched as a whole before they can be labeled. Additionally, other modalities like acceleration data are often recorded alongside a video. For that purpose, we created an annotation tool that enables to annotate datasets of video and inertial sensor data. In contrast to most existing approaches, we focus on semi-supervised labeling support to infer labels for the whole dataset. This means, after labeling a small set of instances our system is able to provide labeling recommendations. We aim to rely on the acceleration data of a wrist-worn sensor to support the labeling of a video recording. For that purpose, we apply template matching to identify time intervals of certain activities. We test our approach on three datasets, one containing warehouse picking activities, one consisting of activities of daily living and one about meal preparations. Our results show that the presented method is able to give hints to annotators about possible label candidates.

scholarly journals A Super-Bagging Method for Volleyball Action Recognition Using Wearable Sensors

2020 ◽  
Vol 4 (2) ◽  
pp. 33 ◽  
Author(s):  
Fasih Haider ◽  
Fahim A. Salim ◽  
Dees B.W. Postma ◽  
Robby van Delden ◽  
Dennis Reidsma ◽  
...  
Keyword(s):  
Wearable Sensors ◽  
Sensor Data ◽  
Dominant Hand ◽  
Imbalanced Learning ◽  
Time Intervals ◽  
Video Recordings ◽  
Potential Applications ◽  
Individual Player ◽  
Measurement Units ◽  
Bagging Method

Access to performance data during matches and training sessions is important for coaches and players. Although there are many video tagging systems available which can provide such access, these systems require manual effort. Data from Inertial Measurement Units (IMU) could be used for automatically tagging video recordings in terms of players’ actions. However, the data gathered during volleyball sessions are generally very imbalanced, since for an individual player most time intervals can be classified as “non-actions” rather than “actions”. This makes automatic annotation of video recordings of volleyball matches a challenging machine-learning problem. To address this problem, we evaluated balanced and imbalanced learning methods with our newly proposed ‘super-bagging’ method for volleyball action modelling. All methods are evaluated using six classifiers and four sensors (i.e., accelerometer, magnetometer, gyroscope and barometer). We demonstrate that imbalanced learning provides better unweighted average recall, (UAR = 83.99%) for the non-dominant hand using a naive Bayes classifier than balanced learning, while balanced learning provides better performance (UAR = 84.18%) for the dominant hand using a tree bagger classifier than imbalanced learning. Our super-bagging method provides the best UAR (84.19%). It is also noted that the super-bagging method provides better averaged UAR than balanced and imbalanced methods in 8 out of 10 cases, hence demonstrating the potential of the super-bagging method for IMU’s sensor data. One of the potential applications of these novel models is fatigue and stamina estimation e.g., by keeping track of how many actions a player is performing and when these are being performed.

scholarly journals Evaluation of Inertial Sensor Data by a Comparison with Optical Motion Capture Data of Guitar Strumming Gestures

Sensors ◽  
2020 ◽  
Vol 20 (19) ◽  
pp. 5722
Author(s):  
Sérgio Freire ◽  
Geise Santos ◽  
Augusto Armondes ◽  
Eduardo A. L. Meneses ◽  
Marcelo M. Wanderley
Keyword(s):  
Motion Capture ◽  
Inertial Sensor ◽  
Low Cost ◽  
Sensor Data ◽  
Level Control ◽  
Interactive Music ◽  
Acceleration Data ◽  
Optical Motion ◽  
Optical Motion Capture ◽  
High Level

Computing technologies have opened up a myriad of possibilities for expanding the sonic capabilities of acoustic musical instruments. Musicians nowadays employ a variety of rather inexpensive, wireless sensor-based systems to obtain refined control of interactive musical performances in actual musical situations like live music concerts. It is essential though to clearly understand the capabilities and limitations of such acquisition systems and their potential influence on high-level control of musical processes. In this study, we evaluate one such system composed of an inertial sensor (MetaMotionR) and a hexaphonic nylon guitar for capturing strumming gestures. To characterize this system, we compared it with a high-end commercial motion capture system (Qualisys) typically used in the controlled environments of research laboratories, in two complementary tasks: comparisons of rotational and translational data. For the rotations, we were able to compare our results with those that are found in the literature, obtaining RMSE below 10° for 88% of the curves. The translations were compared in two ways: by double derivation of positional data from the mocap and by double integration of IMU acceleration data. For the task of estimating displacements from acceleration data, we developed a compensative-integration method to deal with the oscillatory character of the strumming, whose approximative results are very dependent on the type of gestures and segmentation; a value of 0.77 was obtained for the average of the normalized covariance coefficients of the displacement magnitudes. Although not in the ideal range, these results point to a clearly acceptable trade-off between the flexibility, portability and low cost of the proposed system when compared to the limited use and cost of the high-end motion capture standard in interactive music setups.

Effects of Time and Perceptual Orientation on Actors' and Observers' Attributions

1984 ◽  
Vol 58 (1) ◽  
pp. 23-30
Author(s):  
Donald S. Martin ◽  
Ming-Shiunn Huang
Keyword(s):  
Video Recording ◽  
Control Group ◽  
Control Groups ◽  
Video Recordings ◽  
Before And After ◽  
Perceptual Orientation ◽  
Initial Difference ◽  
Variety Show ◽  
And Control ◽  
Unrelated Variety

The actor/observer effect was examined by Storms in a 1973 study which manipulated perceptual orientation using video recordings. Storms' study was complex and some of his results equivocal. The present study attempted to recreate the perceptual reorientation effect using a simplified experimental design and an initial difference between actors and observers which was the reverse of the original effect. Female undergraduates performed a motor co-ordination task as actors while watched by observers. Each person made attributions for the actor's behaviour before and after watching a video recording of the performance. For a control group the video recording was of an unrelated variety show excerpt. Actors' initial attributions were less situational than observers'. Both actors and observers became more situational after the video replay but this effect occurred in both experimental and control groups. It was suggested the passage of time between first and second recording of attributions could account for the findings and care should be taken when interpreting Storms' (1973) study and others which did not adequately control for temporal effects.

QUALITATIVE CHANGES IN RHYTHMIC GYMNASTICS HOOP THROWS FROM THE PERSPECTIVE OF BIOMECHANICAL ANALYSIS

2020 ◽  
pp. 1-8
Author(s):  
Raluca Tanasa
Keyword(s):  
Correlation Coefficient ◽  
Video Recording ◽  
Biomechanical Analysis ◽  
Fundamental Groups ◽  
Horizontal Distance ◽  
Video Recordings ◽  
Rhythmic Gymnastics ◽  
Female Gymnasts ◽  
Execution Technique ◽  
Qualitative Changes

Throws and catches in rhythmic gymnastics represent one of the fundamental groups of apparatus actuation. They represent for the hoop actions of great showmanship, but also elements of risk. The purpose of this paper is to improve the throw execution technique through biomechanical analysis in order to increase the performance of female gymnasts in competitions. The subjects of this study were 8 gymnasts aged 9-10 years old, practiced performance Rhythmic Gymnastics. The experiment consisted in video recording and the biomechanical analysis of the element “Hoop throw, step jump and catch”. After processing the video recordings using the Simi Motion software, we have calculated and obtained values concerning: launch height, horizontal distance and throwing angle between the arm and the horizontal. Pursuant to the data obtained, we have designed a series of means to improve the execution technique for the elements comprised within the research and we have implemented them in the training process. Regarding the interpretation of the results, it may be highlighted as follows: height and horizontal distance in this element have values of the correlation coefficient of 0.438 and 0.323, thus a mean significance of 0.005. The values of the arm/horizontal angle have improved for all the gymnasts, the correlation coefficient being 0.931, with a significance of 0.01. As a general conclusion, after the results obtained, it may be stated that the means introduced in the experiment have proven their efficacy, which has led to the optimisation of the execution technique, thus confirming the research hypothesis.

Spatial-temporal variables for swimming coaches: A comparison study between video and TritonWear sensor

2021 ◽  
pp. 174795412110137
Author(s):  
Robin Pla ◽  
Thibaut Ledanois ◽  
Escobar David Simbana ◽  
Anaël Aubry ◽  
Benjamin Tranchard ◽  
...  
Keyword(s):  
Swimming Performance ◽  
Video Recording ◽  
Open Water ◽  
Percentage Error ◽  
Stroke Index ◽  
Wearable Sensor ◽  
Stroke Length ◽  
Video Recordings ◽  
Temporal Variables ◽  
Training Sets

The main aim of this study was to evaluate the validity and the reliability of a swimming sensor to assess swimming performance and spatial-temporal variables. Six international male open-water swimmers completed a protocol which consisted of two training sets: a 6×100m individual medley and a continuous 800 m set in freestyle. Swimmers were equipped with a wearable sensor, the TritonWear to collect automatically spatial-temporal variables: speed, lap time, stroke count (SC), stroke length (SL), stroke rate (SR), and stroke index (SI). Video recordings were added as a “gold-standard” and used to assess the validity and the reliability of the TritonWear sensor. The results show that the sensor provides accurate results in comparison with video recording measurements. A very high accuracy was observed for lap time with a mean absolute percentage error (MAPE) under 5% for each stroke (2.2, 3.2, 3.4, 4.1% for butterfly, backstroke, breaststroke and freestyle respectively) but high error ranges indicate a dependence on swimming technique. Stroke count accuracy was higher for symmetric strokes than for alternate strokes (MAPE: 0, 2.4, 7.1 & 4.9% for butterfly, breaststroke, backstroke & freestyle respectively). The other variables (SL, SR & SI) derived from the SC and the lap time also show good accuracy in all strokes. The wearable sensor provides an accurate real time feedback of spatial-temporal variables in six international open-water swimmers during classical training sets (at low to moderate intensities), which could be a useful tool for coaches, allowing them to monitor training load with no effort.

scholarly journals On Expressive Features for Gait Analysis using Lower Limb Inertial Sensor Data

2020 ◽  
Vol 53 (2) ◽  
pp. 15990-15997
Author(s):  
Felix Laufer ◽  
Michael Lorenz ◽  
Bertram Taetz ◽  
Gabriele Bleser
Keyword(s):  
Gait Analysis ◽  
Lower Limb ◽  
Inertial Sensor ◽  
Sensor Data

scholarly journals Interpretable deep learning for the remote characterisation of ambulation in multiple sclerosis using smartphones

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Andrew P. Creagh ◽  
Florian Lipsmeier ◽  
Michael Lindemann ◽  
Maarten De Vos
Keyword(s):  
Multiple Sclerosis ◽  
Deep Learning ◽  
Inertial Sensor ◽  
Heterogeneous Data ◽  
Fine Tuning ◽  
Sensor Data ◽  
Support Vector ◽  
Deep Convolutional Neural Networks ◽  
Healthcare Applications ◽  
Feature Based

AbstractThe emergence of digital technologies such as smartphones in healthcare applications have demonstrated the possibility of developing rich, continuous, and objective measures of multiple sclerosis (MS) disability that can be administered remotely and out-of-clinic. Deep Convolutional Neural Networks (DCNN) may capture a richer representation of healthy and MS-related ambulatory characteristics from the raw smartphone-based inertial sensor data than standard feature-based methodologies. To overcome the typical limitations associated with remotely generated health data, such as low subject numbers, sparsity, and heterogeneous data, a transfer learning (TL) model from similar large open-source datasets was proposed. Our TL framework leveraged the ambulatory information learned on human activity recognition (HAR) tasks collected from wearable smartphone sensor data. It was demonstrated that fine-tuning TL DCNN HAR models towards MS disease recognition tasks outperformed previous Support Vector Machine (SVM) feature-based methods, as well as DCNN models trained end-to-end, by upwards of 8–15%. A lack of transparency of “black-box” deep networks remains one of the largest stumbling blocks to the wider acceptance of deep learning for clinical applications. Ensuing work therefore aimed to visualise DCNN decisions attributed by relevance heatmaps using Layer-Wise Relevance Propagation (LRP). Through the LRP framework, the patterns captured from smartphone-based inertial sensor data that were reflective of those who are healthy versus people with MS (PwMS) could begin to be established and understood. Interpretations suggested that cadence-based measures, gait speed, and ambulation-related signal perturbations were distinct characteristics that distinguished MS disability from healthy participants. Robust and interpretable outcomes, generated from high-frequency out-of-clinic assessments, could greatly augment the current in-clinic assessment picture for PwMS, to inform better disease management techniques, and enable the development of better therapeutic interventions.

scholarly journals Using Machine Learning and Wearable Inertial Sensor Data for the Classification of Fractal Gait Patterns in Women and Men During Load Carriage

2021 ◽  
Vol 185 ◽  
pp. 282-291
Author(s):  
Nizam U. Ahamed ◽  
Kellen T. Krajewski ◽  
Camille C. Johnson ◽  
Adam J. Sterczala ◽  
Julie P. Greeves ◽  
...  
Keyword(s):  
Machine Learning ◽  
Inertial Sensor ◽  
Load Carriage ◽  
Sensor Data ◽  
Gait Patterns

scholarly journals Development and Evaluation of a Low-Drift Inertial Sensor-Based System for Analysis of Alpine Skiing Performance

Sensors ◽  
2021 ◽  
Vol 21 (7) ◽  
pp. 2480
Author(s):  
Isidoro Ruiz-García ◽  
Ismael Navarro-Marchal ◽  
Javier Ocaña-Wilhelmi ◽  
Alberto J. Palma ◽  
Pablo J. Gómez-López ◽  
...  
Keyword(s):  
Data Fusion ◽  
Reference System ◽  
Inertial Sensor ◽  
Low Cost ◽  
Sampling Rate ◽  
Sensor Data ◽  
Fusion Algorithm ◽  
Improve Technique ◽  
Worst Case ◽  
Sensor Data Fusion

In skiing it is important to know how the skier accelerates and inclines the skis during the turn to avoid injuries and improve technique. The purpose of this pilot study with three participants was to develop and evaluate a compact, wireless, and low-cost system for detecting the inclination and acceleration of skis in the field based on inertial measurement units (IMU). To that end, a commercial IMU board was placed on each ski behind the skier boot. With the use of an attitude and heading reference system algorithm included in the sensor board, the orientation and attitude data of the skis were obtained (roll, pitch, and yaw) by IMU sensor data fusion. Results demonstrate that the proposed IMU-based system can provide reliable low-drifted data up to 11 min of continuous usage in the worst case. Inertial angle data from the IMU-based system were compared with the data collected by a video-based 3D-kinematic reference system to evaluate its operation in terms of data correlation and system performance. Correlation coefficients between 0.889 (roll) and 0.991 (yaw) were obtained. Mean biases from −1.13° (roll) to 0.44° (yaw) and 95% limits of agreements from 2.87° (yaw) to 6.27° (roll) were calculated for the 1-min trials. Although low mean biases were achieved, some limitations arose in the system precision for pitch and roll estimations that could be due to the low sampling rate allowed by the sensor data fusion algorithm and the initial zeroing of the gyroscope.

Sensor Fusion Based on Strong Tracking Filter for Augmented Reality Registration

2011 ◽  
Vol 467-469 ◽  
pp. 108-113
Author(s):  
Xin Yu Li ◽  
Dong Yi Chen
Keyword(s):  
Augmented Reality ◽  
Inertial Sensor ◽  
Measurement Data ◽  
Sensor Data ◽  
Data Synchronization ◽  
Error Covariance ◽  
Tightly Coupled ◽  
Strong Tracking Filter ◽  
Fading Factor ◽  
Tracking Filter

Accurate tracking for Augmented Reality applications is a challenging task. Multi-sensors hybrid tracking generally provide more stable than the effect of the single visual tracking. This paper presents a new tightly-coupled hybrid tracking approach combining vision-based systems with inertial sensor. Based on multi-frequency sampling theory in the measurement data synchronization, a strong tracking filter (STF) is used to smooth sensor data and estimate position and orientation. Through adding time-varying fading factor to adaptively adjust the prediction error covariance of filter, this method improves the performance of tracking for fast moving targets. Experimental results show the efficiency and robustness of this proposed approach.

Sign in / Sign up

Export Citation Format

Share Document