scholarly journals Automated Generation of a Digital Twin of a Manufacturing System by Using Scan and Convolutional Neural Networks

2020 ◽  
Author(s):  
Markus Sommer ◽  
Josip Stjepandić ◽  
Sebastian Stobrawa ◽  
Moritz von Soden
Keyword(s):  
Neural Networks ◽  
Object Recognition ◽  
Convolutional Neural Networks ◽  
Shop Floor ◽  
Process Chain ◽  
Automated Generation ◽  
Entire Process ◽  
Digital Twin ◽  
Production Processes ◽  
Novel Approach

The simulation of production processes using a Digital Twin is a promising means for prospective planning, analysis of existing systems or process-parallel monitoring. However, many companies, especially small and medium-sized enterprises, do not apply the technology, because the generation of a Digital Twin is cost-, time- and resource-intensive and IT expertise is required. This obstacle can be removed by a novel approach to generate a Digital Twin using fast scans of the shop floor and subsequent object recognition in the point cloud. We describe how parameters and data should be acquired in order to generate a Digital Twin automatically. An overview of the entire process chain is given. A particular attention is given to the automatic object recognition and its integration into Digital Twin.

scholarly journals Improvement of Factory Planning by Automated Generation of a Digital Twin

2020 ◽  
Author(s):  
Markus Sommer ◽  
Josip Stjepandić ◽  
Sebastian Stobrawa ◽  
Moritz von Soden
Keyword(s):  
Object Recognition ◽  
Production Systems ◽  
Cost Savings ◽  
Cost Effective ◽  
Shop Floor ◽  
Simulation Methods ◽  
Manufacturing Companies ◽  
Factory Planning ◽  
Automated Generation ◽  
Digital Twin

The simulation of processes in production systems is a powerful tool for factory planning. The application of simulation methods within the Digital Factory is becoming increasingly relevant as developments in the field of digitalization lead to more comprehensive, efficient, embedded and cost-effective simulation methods. Especially the integration within a Digital Twin, allows these advantages to be achieved for simulations. Here, the Digital Twin can be utilized for prospective planning, analysis of existing systems or process-oriented monitoring. In all cases, the Digital Twin offers manufacturing companies room for improvement in production and logistics processes leading to cost savings. However, many companies do not apply the technology, because the generation of a Digital Twin is cost-, time- and resource-intensive and IT expertise is required. This paper presents an approach for generating a Digital Twin in the built environment automatically and for utilization in factory planning. The obstacles will be overcome by using a scan of the shop floor, subsequent object recognition, and predefined frameworks for factory planning within the Digital Twin. Here, the effort for scanning the production hall is additional, while the subsequent object recognition, the generation of the CAD model and the simulation model, as well as the factory planning can be to a great extent automated and therefore carried out with a minimum of effort. Therefore, considerable cost savings can be expected here, which more than offset the additional effort for scanning.

scholarly journals Human and bird detection and classification based on Doppler radar spectrograms and vision images using convolutional neural networks

2021 ◽  
Vol 18 (3) ◽  
pp. 172988142110105
Author(s):  
Jnana Sai Abhishek Varma Gokaraju ◽  
Weon Keun Song ◽  
Min-Ho Ka ◽  
Somyot Kaitwanidvilai
Keyword(s):  
Neural Networks ◽  
Object Recognition ◽  
Convolutional Neural Networks ◽  
Doppler Radar ◽  
Autonomous Vehicle ◽  
Sampling Point ◽  
Body Segment ◽  
Kinematic Modeling ◽  
Deep Convolutional Neural Networks ◽  
Kinematic Models

The study investigated object detection and classification based on both Doppler radar spectrograms and vision images using two deep convolutional neural networks. The kinematic models for a walking human and a bird flapping its wings were incorporated into MATLAB simulations to create data sets. The dynamic simulator identified the final position of each ellipsoidal body segment taking its rotational motion into consideration in addition to its bulk motion at each sampling point to describe its specific motion naturally. The total motion induced a micro-Doppler effect and created a micro-Doppler signature that varied in response to changes in the input parameters, such as varying body segment size, velocity, and radar location. Micro-Doppler signature identification of the radar signals returned from the target objects that were animated by the simulator required kinematic modeling based on a short-time Fourier transform analysis of the signals. Both You Only Look Once V3 and Inception V3 were used for the detection and classification of the objects with different red, green, blue colors on black or white backgrounds. The results suggested that clear micro-Doppler signature image-based object recognition could be achieved in low-visibility conditions. This feasibility study demonstrated the application possibility of Doppler radar to autonomous vehicle driving as a backup sensor for cameras in darkness. In this study, the first successful attempt of animated kinematic models and their synchronized radar spectrograms to object recognition was made.

scholarly journals RGB-D-Based Object Recognition Using Multimodal Convolutional Neural Networks: A Survey

IEEE Access ◽  
2019 ◽  
Vol 7 ◽  
pp. 43110-43136 ◽  
Author(s):  
Mingliang Gao ◽  
Jun Jiang ◽  
Guofeng Zou ◽  
Vijay John ◽  
Zheng Liu
Keyword(s):  
Neural Networks ◽  
Object Recognition ◽  
Convolutional Neural Networks

scholarly journals Leveraging Graphical User Interface Automation for Generic Robot Programming

Robotics ◽  
2020 ◽  
Vol 10 (1) ◽  
pp. 3
Author(s):  
Tudor B. Ionescu
Keyword(s):  
User Interface ◽  
Graphical User Interface ◽  
Programming Models ◽  
Shop Floor ◽  
Robot Programming ◽  
User Interactions ◽  
Programming Environments ◽  
Automated Generation ◽  
Novel Approach ◽  
Block Based

A novel approach to generic (or generalized) robot programming and a novel simplified, block-based programming environment, called “Assembly”, are introduced. The approach leverages the newest graphical user interface automation tools and techniques to generate programs in various proprietary robot programming environments by emulating user interactions in those environments. The “Assembly” tool is used to generate robot-independent intermediary program models, which are translated into robot-specific programs using a graphical user interface automation toolchain. The generalizability of the approach to list, tree, and block-based programming is assessed using three different robot programming environments, two of which are proprietary. The results of this evaluation suggest that the proposed approach is feasible for an entire range of programming models and thus enables the generation of programs in various proprietary robot programming environments. In educational settings, the automated generation of programs fosters learning different robot programming models by example. For experts, the proposed approach provides a means for generating program (or task) templates, which can be adjusted to the needs of the application at hand on the shop floor.

TAD-Net: An approach for real-time action detection based on temporal convolution network and graph convolution network in digital twin shop-floor

Digital Twin ◽  
2021 ◽  
Vol 1 ◽  
pp. 10
Author(s):  
Qing Hong ◽  
Yifeng Sun ◽  
Tingyu Liu ◽  
Liang Fu ◽  
Yunfeng Xie
Keyword(s):  
Real Time ◽  
Sequence Data ◽  
Network Models ◽  
Human Action ◽  
Practical Significance ◽  
Shop Floor ◽  
Action Detection ◽  
Digital Twin ◽  
Actual Application ◽  
Production Processes

Background: Intelligent monitoring of human action in production is an important step to help standardize production processes and construct a digital twin shop-floor rapidly. Human action has a significant impact on the production safety and efficiency of a shop-floor, however, because of the high individual initiative of humans, it is difficult to realize real-time action detection in a digital twin shop-floor. Methods: We proposed a real-time detection approach for shop-floor production action. This approach used the sequence data of continuous human skeleton joints sequences as the input. We then reconstructed the Joint Classification-Regression Recurrent Neural Networks (JCR-RNN) based on Temporal Convolution Network (TCN) and Graph Convolution Network (GCN). We called this approach the Temporal Action Detection Net (TAD-Net), which realized real-time shop-floor production action detection. Results: The results of the verification experiment showed that our approach has achieved a high temporal positioning score, recognition speed, and accuracy when applied to the existing Online Action Detection (OAD) dataset and the Nanjing University of Science and Technology 3 Dimensions (NJUST3D) dataset. TAD-Net can meet the actual needs of the digital twin shop-floor. Conclusions: Our method has higher recognition accuracy, temporal positioning accuracy, and faster running speed than other mainstream network models, it can better meet actual application requirements, and has important research value and practical significance for standardizing shop-floor production processes, reducing production security risks, and contributing to the understanding of real-time production action.

Sign in / Sign up

Export Citation Format

Share Document