Classification of Assembly Operations Using Recurrent Neural Networks

Visual sensor data of manual assembly operations offers rich information that can be extracted in order to analyze and digitalize the assembly. The worker’s interaction with tools and objects, as well as the spatial–temporal nature of assembly operations, makes the recognition and classification of assembly operations a complex task. Therefore, classical methods of computer vision do not provide a sufficient solution. This paper presents a recurrent neural network for the classification of manual assembly operations using visual sensor data and addresses the question as to what extent such a solution is feasible in terms of robustness and reliability. Since complex assembly operations are a combination of basic movements, four main assembly operations of the Methods Time-Measurement base operations are classified using a machine learning approach. A dataset of these four assembly operations, reach, grasp, move and release, containing RGB-, infrared-, and depth-data is used. A Convolutional Neural Network—Long Short Term Memory architecture is investigated regarding its applicability due to the spatial–temporal nature of the data.

Knowledge extract and ontology construction method of assembly process text

The actual product assembly process mainly relies on manual assembly by workers, and the personal experience of workers is difficult to effectively reuse. Ontology as a knowledge management and expression tool is gradually applied in the field of assembly. However, the manual construction of the ontology is time-consuming and labor-intensive, and the automatic construction of the ontology requires a large number of corpora for training, both of which are difficult to obtain a good assembly case ontology. This paper proposes a method in which automatically extracts relevant knowledge from case assembly process files to generates case database and integrates ontology framework of assembly domain to construct ontology. It shows that the accuracy can be guaranteed on the basis of the rapid construction of case ontology. The feasibility of this method is proved by a practical case.

ModElec

Integrating electronics with highly custom 3D designs for the physical fabrication of interactive prototypes is traditionally cumbersome and requires numerous iterations of manual assembly and debugging. With the new capabilities of 3D printers, combining electronic design and 3D modeling workflows can lower the barrier for achieving interactive functionality or iterating on the overall design. We present ModElec---an interactive design tool that enables the coordinated expression of electronic and physical design intent by allowing designers to integrate 3D-printable circuits with 3D forms. With ModElec, the user can arrange electronic parts in a 3D body, modify the model design with embedded circuits updated, and preview the auto-generated 3D traces that can be directly printed with a multi-material-based 3D printer. We demonstrate the potential of ModElec with four example applications, from a set of game controls to reconfigurable devices. Further, the tool was reported as easy to use through a preliminary evaluation with eight designers.

Continuous ergonomic risk perception for manual assembly operations using wearable multi-sensor posture estimation

Purpose This paper aims to perform the real-time and accurate ergonomics analysis for the operator in the manual assembly, with the purpose of identifying potential ergonomic injuries when encountering labor-excessive and unreasonable assembly operations. Design/methodology/approach Instead of acquiring body data for ergonomic evaluation by arranging many observers around, this paper proposes a multi-sensor based wearable system to track worker’s posture for a continuous ergonomic assessment. Moreover, given the accurate neck postural data from the shop floor by the proposed wearable system, a continuous rapid upper limb assessment method with robustness to occasional posture changes, is proposed to evaluate the neck and upper back risk during the manual assembly operations. Findings The proposed method can retrieve human activity data during manual assembly operations, and experimental results illustrate that the proposed work is flexible and accurate for continuous ergonomic assessments in manual assembly operations. Originality/value Based on the proposed multi-sensor based wearable system for posture acquisition, a real-time and high-precision ergonomics analysis is achieved with the postural data arrived continuously, it can provide a more objective indicator to assess the ergonomics during manual assembly.

A deep learning-based worker assistance system for error prevention: Case study in a real-world manual assembly

Modern assembly systems adapt to the requirements of customised and short-lived products. As assembly tasks become increasingly complex and change rapidly, the cognitive load on employees increases. This leads to the use of assistance systems for manual assembly to detect and avoid human errors and thus ensure consistent product quality. Most of these systems promise to improve the production environment but have hardly been studied quantitatively so far. Recent advances in deep learning-based computer vision have also not yet been fully exploited. This study aims to provide architectural, and implementational details of a state-of-the-art assembly assistance system based on an object detection model. The proposed architecture is intended to be representative of modern assistance systems. The error prevention potential is determined in a case study in which test subjects manually assemble a complex explosion-proof tubular lamp. The results show 51 % fewer assembly errors compared to a control group without assistance. Three of the four considered types of error classes have been reduced by at least 42 %. In particular, errors by omission are most likely to be prevented by the system. The reduction in the error rate is observed over the entire period of 30 consecutive product assemblies, comparing assisted and unassisted assembly. Furthermore, the recorded assembly data are found to be valuable regarding traceability and production improvement processes.

The impact of the collaborative workplace on the production system capacity: Simulation modelling vs. real-world application approach

In recent years, there have been more and more collaborative workplaces in different types of manufacturing systems. Although the introduction of collaborative workplaces can be cost-effective, there is still much uncertainty about how such workplaces affect the capacity of the rest of production system. The article presents the importance of introducing collaborative workplaces in manual assembly operations where the production capacities are already limited. With the simulation modelling method, the evaluation of the introduction impact of collaborative workplaces on manual assembly operations that represent bottlenecks in the production process is presented. The research presents two approaches to workplace performance evaluation, both simulation modelling and a real-world collaborative workplace example, as a basis of a detailed time study. The main findings are comparisons of simulation modelling results and a study of a real-world collaborative workplace, with graphically and numerically presented parameters describing the utilization of production capacities, their efficiency and financial justification. The research confirms the expediency of the collaborative workplaces use and emphasise the importance of further research in the field of their technological and sociological impacts.