scholarly journals Agricultural Machinery Virtual Assembly System Using Dynamic Gesture Recognitive Interaction Based on a CNN and LSTM Network

2021 ◽  
Vol 2021 ◽  
pp. 1-16
Author(s):  
Po Zhang ◽  
Junqiang Lin ◽  
Jianhua He ◽  
Xiuchan Rong ◽  
Chengen Li ◽  
...  
Keyword(s):  
Virtual Reality ◽  
Gesture Recognition ◽  
Crop Production ◽  
Vision System ◽  
Virtual Assembly ◽  
Assembly System ◽  
Leap Motion ◽  
Agricultural Machinery ◽  
Lstm Network ◽  
Dynamic Gesture Recognition

The agricultural machinery experiment is restricted by the crop production season. Missing the crop growth cycle will extend the machine development period. The use of virtual reality technology to complete preassembly and preliminary experiments can reduce the loss caused by this problem. To improve the intelligence and stability of virtual assembly, this paper proposed a more stable dynamic gesture cognition framework: the TCP/IP protocol constituted the network communication terminal, the leap motion-based vision system constituted the gesture data collection terminal, and the CNN-LSTM network constituted the dynamic gesture recognition classification terminal. The dynamic gesture recognition framework and the harvester virtual assembly platform formed a virtual assembly system to achieve gesture interaction. Through experimental analysis, the improved CNN-LSTM network had a small volume and could quickly establish a stable and accurate gesture recognition model with an average accuracy of 98.0% (±0.894). The assembly efficiency of the virtual assembly system with the framework was improved by approximately 15%. The results showed that the accuracy and stability of this model met the requirements, the corresponding assembly parts were robust in the virtual simulation environment of the whole machine, and the harvesting behaviour in the virtual reality scene was close to the real scene. The virtual assembly system under this framework provided technical support for unmanned farms and virtual experiments on agricultural machinery.

scholarly journals Recognition of American Sign Language Gestures in a Virtual Reality Using Leap Motion

2019 ◽  
Vol 9 (3) ◽  
pp. 445 ◽  
Author(s):  
Aurelijus Vaitkevičius ◽  
Mantas Taroza ◽  
Tomas Blažauskas ◽  
Robertas Damaševičius ◽  
Rytis Maskeliūnas ◽  
...  
Keyword(s):  
Virtual Reality ◽  
American Sign Language ◽  
Sign Language ◽  
Gesture Recognition ◽  
Three Dimensional ◽  
Leap Motion ◽  
American Sign ◽  
3D Space

We perform gesture recognition in a Virtual Reality (VR) environment using dataproduced by the Leap Motion device. Leap Motion generates a virtual three-dimensional (3D) handmodel by recognizing and tracking user‘s hands. From this model, the Leap Motion applicationprogramming interface (API) provides hand and finger locations in the 3D space. We present asystem that is capable of learning gestures by using the data from the Leap Motion device and theHidden Markov classification (HMC) algorithm. We have achieved the gesture recognitionaccuracy (mean ± SD) is 86.1 ± 8.2% and gesture typing speed is 3.09 ± 0.53 words per minute(WPM), when recognizing the gestures of the American Sign Language (ASL).

Research on Virtual Assembly Platform Realization Technology of Vehicle Transmission System

2011 ◽  
Vol 135-136 ◽  
pp. 856-861 ◽  
Author(s):  
Fei Liu ◽  
Qing Dong Yan ◽  
Shou Wen Yao
Keyword(s):  
Virtual Reality ◽  
Virtual Assembly ◽  
Transmission System ◽  
Assembly System ◽  
Requirement Analysis ◽  
Virtual Reality System ◽  
Hardware System ◽  
Key Technologies ◽  
Software And Hardware

The necessity of constructing a virtual assembly platform for vehicle transmission is discussed in this paper. Based on function requirement analysis of virtual assembly system for vehicle transmission, the architecture of virtual assembly platform is established. Detailed functions of platform are given. Virtual reality system and structure of software and hardware system for the virtual assembly platform are introduced in detail. The key technologies of virtual assembly platform have been expounded. Some practical instances are realized for verifying relevant functions of the virtual assembly platform of vehicle transmission system.

scholarly journals Research and Application of Multifeature Gesture Recognition in Human-Computer Interaction Based on Virtual Reality Technology

2021 ◽  
Vol 2021 ◽  
pp. 1-5
Author(s):  
Zhao Feng ◽  
Jinlong Wu ◽  
Taile Ni
Keyword(s):  
Virtual Reality ◽  
Human Computer Interaction ◽  
Gesture Recognition ◽  
Central Region ◽  
Feature Recognition ◽  
Feature Fusion ◽  
Time Dynamic ◽  
Information Recognition ◽  
Dynamic Gesture Recognition ◽  
Computer Interaction

Objective. To explore the research and application of multifeature gesture recognition in virtual reality human-computer interaction and to explore the gesture recognition technology scheme to achieve better human-computer interaction experience. Methods. Through the study of the technical difficulties of gesture recognition, comparative static gesture feature recognition and feature fusion algorithms are applied, in the process of research on gesture partition, and adjust the contrast of characteristic parameters, combined with the feature of space-time dynamic gesture tracking trajectory and dynamic gesture recognition and gesture recognition effect under different scheme. Results. The central region was divided into 0 regions, and the central region was divided into 1-4 regions in counterclockwise direction. Compared with the traditional gesture changes, the overlapping problem in the four partition modes was reduced, the gesture was better displayed, and the operation and use of gesture processing were realized more efficiently. Conclusion. Gesture recognition requires the combination of static gesture feature information recognition, gesture feature fusion, spatiotemporal trajectory feature, and dynamic gesture trajectory feature to achieve a better human-computer interaction experience.

Investigation on Key Technology of Virtual Assembly and Construction for Tunnel Boring Machine

2011 ◽  
Vol 71-78 ◽  
pp. 3375-3378 ◽  
Author(s):  
Jun Cheng ◽  
Ya Dong Gong ◽  
Yue Ming Liu
Keyword(s):  
Virtual Reality ◽  
Virtual Assembly ◽  
Tunnel Boring Machine ◽  
Simulation System ◽  
Virtual Simulation ◽  
Assembly System ◽  
Application Program ◽  
Virtual Construction ◽  
Tunnel Boring ◽  
Shield Machine

This study presents a practical requirement and realizes one new virtual construction interactive system for tunnel boring machine. One type of TBM(tunnel boring machine)’s virtual prototype has been built. Software PROE is used to build the shield machine model. Application program of virtual assembly system base on virtual reality is developed by using Division, The author also study on key technologies of constructing virtual assembly system of shield machine, such as interference detection in virtual environment. Application program of virtual simulation system base on virtual reality is developed by using C++. The result shows that the work station can be simulated well, and this virtual simulation system can be extended in virtual training and manipulation, virtual construction and so on.

Sign in / Sign up

Export Citation Format

Share Document