Comparison of optimized Markov Decision Process using Dynamic Programming and Temporal Differencing – A reinforcement learning approach

Reinforcement learning is one of the promising approaches for operations research problems. The incoming inspection process in any manufacturing plant aims to control quality, reduce manufacturing costs, eliminate scrap, and process failure downtimes due to non-conforming raw materials. Prediction of the raw material acceptance rate can regulate the raw material supplier selection and improve the manufacturing process by filtering out non-conformities. This paper presents a Markov model developed to estimate the probability of the raw material being accepted or rejected in an incoming inspection environment. The proposed forecasting model is further optimized for efficiency using the two reinforcement learning algorithms (dynamic programming and temporal differencing). The results of the two optimized models are compared, and the findings are discussed.

Categorization of material quality using a model-free reinforcement learning algorithm

Reinforcement learning is the most preferred algorithms for optimization problems in industrial automation. Model-free reinforcement learning algorithms optimize for rewards without the knowledge of the environmental dynamics and require less computation. Regulating the quality of the raw materials in the inbound inventory can improve the manufacturing process. In this paper, the raw materials arriving at the incoming inspection process are categorized and labeled based on their quality through the path traveled. A model-free temporal difference learning approach is used to predict the acceptance and rejection path of raw materials in the incoming inspection process. The algorithm presented eight routes paths that the raw materials could travel. Four pathways correspond to material acceptance, while the rest lead to material refusal. The materials are annotated using the total scores acquired in the incoming inspection process. The materials traveling on the ideal path (path A) get the highest total score. The rest of the accepted materials in the acceptance path have a 7.37% lower score in path B, whereas path C and path D get 37.28% and 42.44% lower than the ideal approach.

Markov Decision Process approach in the estimation of raw material quality in incoming inspection process

The incoming inspection process in any manufacturing plant aims to control quality, reduce manufacturing costs, eliminate scrap, and process failure downtime due to defective raw materials. Prediction of the raw material acceptance rate can regulate the raw material supplier selection and improve the manufacturing process by filtering out non-conformities. This paper presents a raw material acceptance prediction model (RMAP) developed based on the Markov analysis. RFID tags are used to track the parts throughout the process. A secondary dataset can be derived from the raw RFID data. In this study, a dataset is simulated to reflect a typical incoming inspection process consisting of six substations (Packaging Inspection, Visual Inspection, Gauge Inspection, Rework1, and Rework2) are considered. The accepted parts are forwarded to the Pack and Store station and stored in the warehouse. The non-conforming parts are returned to the supplier. The proposed RMAP model estimates the probability of the raw material being accepted or rejected at each inspection station. The proposed model is evaluated using three test cases: case A (lower conformities), case B (higher conformities) and case C (equal chances of being accepted and rejected). Based on the outcome of the limiting matrix for the three test cases, the results are discussed. The steady-state matrix forecasts the probability of the raw material in a random state. This prediction and forecasting ability of the proposed model enables the industries to save time and cost.

The Effect of Rapid Detection Methods on the Minimization and Prevention of the Risk of Contamination with Aflatoxins in Peanut Products

The methods of analysis and control of aflatoxins in peanuts pursue three key objectives: prevent the entry of contaminated peanuts into ready-to-eat products where they are used as an ingredient; prevent and minimize the risk of cross-contamination from contaminated peanuts to fit-for-use raw materials; perform an appropriate incoming inspection through rapid analysis methods for real-time detection of the absence of or the degree of contamination with aflatoxins. The aim of this study was to analyze the effect of rapid detection methods on the minimization and prevention of the risk of contamination with aflatoxins during the incoming inspection in industries using peanut products in the composition of the finished products. The methods of detection of aflatoxins in peanut products are: Mass Spectrometry combined with High - Performance Liquid Chromatography (HPLC), the internal methodology VAL 92:2010 developed by an accredited laboratory and immunochromatographic rapid tests.

Impact of Raw Material Quality on Performance of Mixing and Charging Machines in Drilling and Blasting Operations

A study into the failure causes of mixing and charging equipment confirms that the main impact on the probability of accidents is the use of raw materials that do not meet the specifications and have unstable properties. The raw materials used for explosives preparation in mechanized charging of boreholes include such components as ammonium nitrate, emulsion phase, diesel fuel, emulsifier and others. The paper describes the application of various formulations with these components in specific types of mixing and charging machines manufactured by AZOTTECH LLC. The main properties that affect the quality of raw materials are summarised, and the incoming inspection of explosive components is described as part of the acceptance procedure at temporary storage sites at a hazardous production facility. The paper describes common types of equipment failures and maintenance procedures when using substandard raw materials. The conclusion highlights the key practices to improve the equipment uptime as well as recommendations for incoming inspection and the use of high-quality explosive components.

Key Figure Based Incoming Inspection of Lithium-Ion Battery Cells

The cell characterization in the incoming inspection is an important but time and cost intensive process step. In order to obtain reliable parameters to evaluate and classify the cells, it is essential to design the test procedures in such a way that the parameters derived from the data allow the required statements about the cells. Before the focus is placed on the evaluation of cell properties, it is therefore necessary to design the test procedures appropriately. In the scope of the investigations two differently designed incoming inspection routines were carried out on 230 commercial lithium-ion battery cells (LIBs) with the aim of deriving recommendations for optimal test procedures. The derived parameters of the test strategies were compared and statistically evaluated. Subsequently, key figures for the classification were identified. As a conclusion, the capacity was confirmed as an already known important parameter and the average cell voltage was identified as a possibility to replace the usually used internal resistance. With regard to capacity, the integration of CV steps in the discharging processes enables the determination independently from the C-rate. For the average voltage cycles with high C-rates are particularly meaningful because of the significant higher scattering due to the overvoltage parts.