Real-Time Depth-Camera Based Hand Tracking for ASL Recognition

A Model-Based System for Real-Time Articulated Hand Tracking Using a Simple Data Glove and a Depth Camera

Sensors ◽

10.3390/s19214680 ◽

2019 ◽

Vol 19 (21) ◽

pp. 4680 ◽

Cited By ~ 1

Author(s):

Linjun Jiang ◽

Hailun Xia ◽

Caili Guo

Keyword(s):

Real Time ◽

Triangular Mesh ◽

Hand Tracking ◽

Depth Camera ◽

Processing Unit ◽

Real Time System ◽

Data Glove ◽

Hand Motion ◽

Mesh Model ◽

Triangular Mesh Model

Tracking detailed hand motion is a fundamental research topic in the area of human-computer interaction (HCI) and has been widely studied for decades. Existing solutions with single-model inputs either require tedious calibration, are expensive or lack sufficient robustness and accuracy due to occlusions. In this study, we present a real-time system to reconstruct the exact hand motion by iteratively fitting a triangular mesh model to the absolute measurement of hand from a depth camera under the robust restriction of a simple data glove. We redefine and simplify the function of the data glove to lighten its limitations, i.e., tedious calibration, cumbersome equipment, and hampering movement and keep our system lightweight. For accurate hand tracking, we introduce a new set of degrees of freedom (DoFs), a shape adjustment term for personalizing the triangular mesh model, and an adaptive collision term to prevent self-intersection. For efficiency, we extract a strong pose-space prior to the data glove to narrow the pose searching space. We also present a simplified approach for computing tracking correspondences without the loss of accuracy to reduce computation cost. Quantitative experiments show the comparable or increased accuracy of our system over the state-of-the-art with about 40% improvement in robustness. Besides, our system runs independent of Graphic Processing Unit (GPU) and reaches 40 frames per second (FPS) at about 25% Central Processing Unit (CPU) usage.

Download Full-text

Real-time and robust hand tracking with a single depth camera

The Visual Computer ◽

10.1007/s00371-013-0894-1 ◽

2013 ◽

Vol 30 (10) ◽

pp. 1133-1144 ◽

Cited By ~ 13

Author(s):

Ziyang Ma ◽

Enhua Wu

Keyword(s):

Real Time ◽

Hand Tracking ◽

Depth Camera

Download Full-text

Real-time Hand Tracking using the RGB-D Camera

International Journal of Computer Graphics ◽

10.21742/ijcg.2017.8.2.03 ◽

2017 ◽

Vol 8 (2) ◽

pp. 13-20 ◽

Cited By ~ 1

Author(s):

Seok-Han Lee ◽

Keyword(s):

Real Time ◽

Hand Tracking

Download Full-text

Data integration by two-sensors in a LEAP-based Virtual Glove for human-system interaction

Multimedia Tools and Applications ◽

10.1007/s11042-020-10296-8 ◽

2021 ◽

Author(s):

Giuseppe Placidi ◽

Danilo Avola ◽

Luigi Cinque ◽

Matteo Polsinelli ◽

Eleni Theodoridou ◽

...

Keyword(s):

Data Integration ◽

Real Time ◽

Vision System ◽

Hand Tracking ◽

Selection Strategy ◽

Motion Sensors ◽

Leap Motion ◽

Time Data ◽

Lower Velocity ◽

Human System

AbstractVirtual Glove (VG) is a low-cost computer vision system that utilizes two orthogonal LEAP motion sensors to provide detailed 4D hand tracking in real–time. VG can find many applications in the field of human-system interaction, such as remote control of machines or tele-rehabilitation. An innovative and efficient data-integration strategy, based on the velocity calculation, for selecting data from one of the LEAPs at each time, is proposed for VG. The position of each joint of the hand model, when obscured to a LEAP, is guessed and tends to flicker. Since VG uses two LEAP sensors, two spatial representations are available each moment for each joint: the method consists of the selection of the one with the lower velocity at each time instant. Choosing the smoother trajectory leads to VG stabilization and precision optimization, reduces occlusions (parts of the hand or handling objects obscuring other hand parts) and/or, when both sensors are seeing the same joint, reduces the number of outliers produced by hardware instabilities. The strategy is experimentally evaluated, in terms of reduction of outliers with respect to a previously used data selection strategy on VG, and results are reported and discussed. In the future, an objective test set has to be imagined, designed, and realized, also with the help of an external precise positioning equipment, to allow also quantitative and objective evaluation of the gain in precision and, maybe, of the intrinsic limitations of the proposed strategy. Moreover, advanced Artificial Intelligence-based (AI-based) real-time data integration strategies, specific for VG, will be designed and tested on the resulting dataset.

Download Full-text

Tomographic Proximity Imaging Using Conductive Sheet for Object Tracking

Sensors ◽

10.3390/s21082736 ◽

2021 ◽

Vol 21 (8) ◽

pp. 2736

Author(s):

Zehao Li ◽

Shunsuke Yoshimoto ◽

Akio Yamamoto

Keyword(s):

Inverse Problem ◽

Theoretical Model ◽

Low Cost ◽

Hand Tracking ◽

Depth Camera ◽

Capacitance Density ◽

Evaluation Test ◽

Density Problem ◽

Imaging Sensor ◽

Tomographic Approach

This paper proposes a proximity imaging sensor based on a tomographic approach with a low-cost conductive sheet. Particularly, by defining capacitance density, physical proximity information is transformed into electric potential. A novel theoretical model is developed to solve the capacitance density problem using the tomographic approach. Additionally, a prototype is built and tested based on the model, and the system solves an inverse problem for imaging the capacitance density change that indicates the object’s proximity change. In the evaluation test, the prototype reaches an error rate of 10.0–15.8% in horizontal localization at different heights. Finally, a hand-tracking demonstration is carried out, where a position difference of 33.8–46.7 mm between the proposed sensor and depth camera is achieved at 30 fps.

Download Full-text