TAD-Net: An approach for real-time action detection based on temporal convolution network and graph convolution network in digital twin shop-floor

Digital Twin ◽  
2021 ◽  
Vol 1 ◽  
pp. 10
Author(s):  
Qing Hong ◽  
Yifeng Sun ◽  
Tingyu Liu ◽  
Liang Fu ◽  
Yunfeng Xie
Keyword(s):  
Real Time ◽  
Sequence Data ◽  
Network Models ◽  
Human Action ◽  
Practical Significance ◽  
Shop Floor ◽  
Action Detection ◽  
Digital Twin ◽  
Actual Application ◽  
Production Processes

Background: Intelligent monitoring of human action in production is an important step to help standardize production processes and construct a digital twin shop-floor rapidly. Human action has a significant impact on the production safety and efficiency of a shop-floor, however, because of the high individual initiative of humans, it is difficult to realize real-time action detection in a digital twin shop-floor. Methods: We proposed a real-time detection approach for shop-floor production action. This approach used the sequence data of continuous human skeleton joints sequences as the input. We then reconstructed the Joint Classification-Regression Recurrent Neural Networks (JCR-RNN) based on Temporal Convolution Network (TCN) and Graph Convolution Network (GCN). We called this approach the Temporal Action Detection Net (TAD-Net), which realized real-time shop-floor production action detection. Results: The results of the verification experiment showed that our approach has achieved a high temporal positioning score, recognition speed, and accuracy when applied to the existing Online Action Detection (OAD) dataset and the Nanjing University of Science and Technology 3 Dimensions (NJUST3D) dataset. TAD-Net can meet the actual needs of the digital twin shop-floor. Conclusions: Our method has higher recognition accuracy, temporal positioning accuracy, and faster running speed than other mainstream network models, it can better meet actual application requirements, and has important research value and practical significance for standardizing shop-floor production processes, reducing production security risks, and contributing to the understanding of real-time production action.

A Digital Twin Concept for Manufacturing Systems

2018 ◽  
Author(s):  
Wesley Ellgass ◽  
Nathan Holt ◽  
Hector Saldana-Lemus ◽  
Julian Richmond ◽  
Ali Vatankhah Barenji ◽  
...  
Keyword(s):  
Virtual Reality ◽  
Real Time ◽  
Manufacturing Systems ◽  
Manufacturing System ◽  
Virtual Space ◽  
Smart Manufacturing ◽  
Shop Floor ◽  
Digital Twin ◽  
Physical Resources ◽  
Virtual Resources

With the developments and applications of the advanced information technologies such as cloud computing, internet of thing, artificial intelligence and virtual reality, industry 4.0 and smart manufacturing era are coming. In this respect, one of the specific challenges is to achieve a connection of physical resources on the shop floor with virtual resources, for real-time response, real time process optimization, and simulation, which is merged by big data problem. In this respect, Digital Twins (DT) concept is introduced as a key technology, which includes physical resources, virtual resources, service system, and digital twin data. DT considers current condition of physical resource and prediction of future events to make a responsive decision. However, due to the complexity of building a digital equivalent in virtual space to its physical counterpart, very little applications have been developed with this purpose, especially in the industrial manufacturing area. Therefore, the types of data and technology required to build the DT for a manufacturing system are presented in this work, trying to develop a framework of DT based manufacturing system, which is supported by the virtual reality for virtualization of physical resources.

How to Establish Digital Thread Using 3D Factory

2019 ◽  
Author(s):  
Holey Ajay ◽  
Alandikar Shashank
Keyword(s):  
Product Design ◽  
Real Time ◽  
Process Planning ◽  
Manufacturing Process ◽  
Shop Floor ◽  
Manufacturing Facility ◽  
Digital Twin ◽  
Manufacturing Process Planning ◽  
Isolated Systems ◽  
Factory Layout

Abstract In a manufacturing assembly line scenario, factory layout is one of the most crucial information used by manufacturing, facility and factory automation engineers for planning purposes. It is important for manufacturing, facility and operations team to work with most up-to-date layout when product, process and operational information on the shop-floor is constantly changing. There are four elements which governs availability of a real-time layout, these are nothing but Product Design, Manufacturing Process Planning, Layout Planning and Shop-floor. The layout must accommodate these changes coming from product design, process updates and shop-floor modifications on real-time basis so that there is no confusion amongst the stakeholders while referring layout data for their planning purpose. If we talk about the impact on the layout because of product design and process design, it is hardly managed real-time due to the isolated systems to manage these data. The integration of product, process and plant (PPP) is becoming crucial to facilitate collaboration and shrink new product introduction lead time where as real-time update from the shop-floor changes is expected in the era of digital transformation. One of the reasons why the integration of product, process and plant (PPP) does not happen is multiple isolated systems used to maintain this data, there are also challenges to feed data back from the shop-floor because of the non-availability of the thread between these objects. The paper is about how factory layout can be developed integrating product, process and plant (PPP) in a single dynamic environment and establish a digital thread between the product design, manufacturing process planning and factory layout to trigger real-time changes and facilitate digital twin of the factory. The methodology adopted here is to develop bill of material for manufacturing resources and align it with the product data management. This approach not only provides ability to maintain change control over resource objects but also helps in configuration management of the resource bill of material. The resources are grouped together as layout structure for the plant with each object required to manufacture the product. The detailed layout developed for the plant while integrating with product and process is used to establish connection with objects on the shop-floor through sensors and IOT (Internet of Things) devices to form digital twin. Such details added in layout which is So far there are no efforts to digitalize every information on the factory floor and able to generate Digital Twin of the factory by connecting physical objects with the digital objects. Paper will elaborate the approach to establish digital thread between PPP and how this can become foundation to drive digital twin of the factory.

The modelling and application of cross-scale human behavior in realizing the shop-floor digital twin

Digital Twin ◽  
2021 ◽  
Vol 1 ◽  
pp. 11
Author(s):  
Tingyu Liu ◽  
Mengming Xia ◽  
Qing Hong ◽  
Yifeng Sun ◽  
Pei Zhang ◽  
...  
Keyword(s):  
Real Time ◽  
Human Behavior ◽  
Manufacturing Industry ◽  
Behavior Modeling ◽  
Shop Floor ◽  
Human Behavior Modeling ◽  
Digital Twin ◽  
Production Behavior ◽  
Time Position ◽  
Human Limb

The digital twin shop-floor has received much attention from the manufacturing industry as it is an important way to upgrade the shop-floor digitally and intelligently. As a key part of the shop-floor, humans' high autonomy and uncertainty leads to the difficulty in digital twin modeling of human behavior. Therefore, the modeling system for cross-scale human behavior in digital twin shop-floors was developed, powered by the data fusion of macro-behavior and micro-behavior virtual models. Shop-floor human macro-behavior mainly refers to the role of the human and their real-time position. Shop-floor micro-behavior mainly refers to real-time human limb posture and production behavior at their workstation. In this study, we reviewed and summarized a set of theoretical systems for cross-scale human behavior modeling in digital twin shop-floors. Based on this theoretical system, we then reviewed modeling theory and technology from macro-behavior and micro-behavior aspects to analyze the research status of shop-floor human behavior modeling. Lastly, we discuss and offer opinion on the application of cross-scale human behavior modeling in digital twin shop-floors. Cross-scale human behavior modeling is the key for realizing closed-loop interactive drive of human behavior in digital twin shop-floors.

Robust Real-Time Human Action Detection through the Fusion of 3D and 2D CNN

2021 ◽  
Author(s):  
Edwin Kwadwo Tenagyei ◽  
Zongbo Hao ◽  
Kwadwo Kusi ◽  
Kwabena Sarpong
Keyword(s):  
Real Time ◽  
Human Action ◽  
Action Detection ◽  
Human Action Detection

scholarly journals Real-Time Production and Logistics Self-Adaption Scheduling Based on Information Entropy Theory

Sensors ◽  
2020 ◽  
Vol 20 (16) ◽  
pp. 4507 ◽  
Author(s):  
Wenchao Yang ◽  
Wenfeng Li ◽  
Yulian Cao ◽  
Yun Luo ◽  
Lijun He
Keyword(s):  
Real Time ◽  
Job Shop ◽  
Job Shop Scheduling ◽  
Cloud Manufacturing ◽  
Practical Significance ◽  
Shop Floor ◽  
Practical Case ◽  
Entropy Model ◽  
Time Model ◽  
The Real

In recent years, the individualized demand of customers brings small batches and diversification of orders towards enterprises. The application of enabling technologies in the factory, such as the industrial Internet of things (IIoT) and cloud manufacturing (CMfg), enhances the ability of customer requirement automatic elicitation and the manufacturing process control. The job shop scheduling problem with a random job arrival time dramatically increases the difficulty in process management. Thus, how to collaboratively schedule the production and logistics resources in the shop floor is very challenging, and it has a fundamental and practical significance of achieving the competitiveness for an enterprise. To address this issue, the real-time model of production and logistics resources is built firstly. Then, the task entropy model is built based on the task information. Finally, the real-time self-adaption collaboration of production and logistics resources is realized. The proposed algorithm is carried out based on a practical case to evaluate its effectiveness. Experimental results show that our proposed algorithm outperforms three existing algorithms.

scholarly journals Real-time Production and Logistics Self-adaption Scheduling based on Information Entropy Theory

2020 ◽  
Author(s):  
Wenchao Yang ◽  
Wenfeng Li ◽  
Yulian Cao ◽  
Yun Luo ◽  
Lijun He
Keyword(s):  
Real Time ◽  
Job Shop ◽  
Job Shop Scheduling ◽  
Cloud Manufacturing ◽  
Practical Significance ◽  
Shop Floor ◽  
Practical Case ◽  
Entropy Model ◽  
Time Model ◽  
The Real

In recent years, the individualized demand of customers brings small batches and diversification of orders towards enterprises. The application of enabling technologies in factory, such as the Industrial Internet of Things (IIoT) and Cloud Manufacturing (CMfg), enhances the ability of customer requirement automatic elicitation and the manufacturing process control. The job shop scheduling problem with random job arrival time dramatically increases the difficulty in process management. Thus, how to collaboratively schedule the production and logistics resources in the shop floor is very challenging, and it has a fundamental and practical significance of achieving the competitiveness for an enterprise. To address this issue, the real-time model of production and logistics resources is built firstly. Then, the task entropy model is built based on the task information. Finally, the real-time self-adaption collaboration of production and logistics resources is realized. The proposed algorithm is carried out based on a practical case to evaluate its effectiveness. Experimental results show that our proposed algorithm outperforms three existing algorithms.

scholarly journals Automated Generation of a Digital Twin of a Manufacturing System by Using Scan and Convolutional Neural Networks

2020 ◽  
Author(s):  
Markus Sommer ◽  
Josip Stjepandić ◽  
Sebastian Stobrawa ◽  
Moritz von Soden
Keyword(s):  
Neural Networks ◽  
Object Recognition ◽  
Convolutional Neural Networks ◽  
Shop Floor ◽  
Process Chain ◽  
Automated Generation ◽  
Entire Process ◽  
Digital Twin ◽  
Production Processes ◽  
Novel Approach

The simulation of production processes using a Digital Twin is a promising means for prospective planning, analysis of existing systems or process-parallel monitoring. However, many companies, especially small and medium-sized enterprises, do not apply the technology, because the generation of a Digital Twin is cost-, time- and resource-intensive and IT expertise is required. This obstacle can be removed by a novel approach to generate a Digital Twin using fast scans of the shop floor and subsequent object recognition in the point cloud. We describe how parameters and data should be acquired in order to generate a Digital Twin automatically. An overview of the entire process chain is given. A particular attention is given to the automatic object recognition and its integration into Digital Twin.

CONTROL IN REAL TIME THE DENSITY OF PRODUCT AT VARIOUS STAGES OF THE TECHNOLOGICAL PROCESS OF SUGAR PRODUCTION

2020 ◽  
Author(s):  
S.B. Kudryashev ◽  
◽  
N.S. Assev ◽  
R.D. Belashov ◽  
V.A. Naumenko ◽  
...  
Keyword(s):  
Real Time ◽  
Technological Process ◽  
Sugar Industry ◽  
Sugar Production ◽  
Production Processes ◽  
Quality Of Products

The article is devoted to solving one of the most important problems of the development of the sugar industry in Russia – the modernization of sugar production processes. Today, sugar production is actively being modernized, shifting most of its processes to the path of avomatization and optimization to improve the quality of products. This article describes one of the main ways to obtain information about the concentration of sucrose in syrup in the production of sugar.

Sign in / Sign up

Export Citation Format

Share Document