scholarly journals A deep q-learning-based optimization of the inventory control in a linear process chain

2020 ◽  
Author(s):  
M.-A. Dittrich ◽  
S. Fohlmeister
Keyword(s):  
Inventory Control ◽  
Data Exchange ◽  
Linear Process ◽  
Process Chain ◽  
Production Networks ◽  
Initial State ◽  
Q Learning ◽  
Reward Function ◽  
Decision Optimization ◽  
Process Chains

AbstractDue to growing globalized markets and the resulting globalization of production networks across different companies, inventory and order optimization is becoming increasingly important in the context of process chains. Thus, an adaptive and continuously self-optimizing inventory control on a global level is necessary to overcome the resulting challenges. Advances in sensor and communication technology allow companies to realize a global data exchange to achieve a holistic inventory control. Based on deep q-learning, a method for a self-optimizing inventory control is developed. Here, the decision process is based on an artificial neural network. Its input is modeled as a state vector that describes the current stocks and orders within the process chain. The output represents a control vector that controls orders for each individual station. Furthermore, a reward function, which is based on the resulting storage and late order costs, is implemented for simulations-based decision optimization. One of the main challenges of implementing deep q-learning is the hyperparameter optimization for the training process, which is investigated in this paper. The results show a significant sensitivity for the leaning rate α and the exploration rate ε. Based on optimized hyperparameters, the potential of the developed methodology could be shown by significantly reducing the total costs compared to the initial state and by achieving stable control behavior for a process chain containing up to 10 stations.

scholarly journals Integrating Production Planning with Truck-Dispatching Decisions through Reinforcement Learning While Managing Uncertainty

Minerals ◽  
2021 ◽  
Vol 11 (6) ◽  
pp. 587
Author(s):  
Joao Pedro de Carvalho ◽  
Roussos Dimitrakopoulos
Keyword(s):  
Reinforcement Learning ◽  
Discrete Event ◽  
Mining Operations ◽  
Fixed Sequence ◽  
Q Learning ◽  
Reward Function ◽  
Copper Gold ◽  
Mining Complex ◽  
Learning Reinforcement ◽  
Operational Plan

This paper presents a new truck dispatching policy approach that is adaptive given different mining complex configurations in order to deliver supply material extracted by the shovels to the processors. The method aims to improve adherence to the operational plan and fleet utilization in a mining complex context. Several sources of operational uncertainty arising from the loading, hauling and dumping activities can influence the dispatching strategy. Given a fixed sequence of extraction of the mining blocks provided by the short-term plan, a discrete event simulator model emulates the interaction arising from these mining operations. The continuous repetition of this simulator and a reward function, associating a score value to each dispatching decision, generate sample experiences to train a deep Q-learning reinforcement learning model. The model learns from past dispatching experience, such that when a new task is required, a well-informed decision can be quickly taken. The approach is tested at a copper–gold mining complex, characterized by uncertainties in equipment performance and geological attributes, and the results show improvements in terms of production targets, metal production, and fleet management.

Intrinsic Hybrid Composites for Lightweight Structures: New Process Chain Approaches

2016 ◽  
Vol 1140 ◽  
pp. 239-246 ◽  
Author(s):  
Simon Frederik Koch ◽  
Daniel Barfuss ◽  
Mathias Bobbert ◽  
Lukas Groß ◽  
Raik Grützner ◽  
...  
Keyword(s):  
Hybrid Composites ◽  
Process Chain ◽  
Lightweight Structures ◽  
Fiber Placement ◽  
Reinforced Plastics ◽  
Steel Sheets ◽  
Process Chains ◽  
New Process ◽  
Hybrid Laminates ◽  
Automated Fiber Placement

This publication describes new process chain approaches for the manufacturing of intrinsic hybrid composites for lightweight structures. The introduced process chains show a variety of different part and sample types, like insert technology for fastening of hollow hybrid shafts and profiles. Another field of research are hybrid laminates with different layers of carbon fiber reinforced plastics stacked with aluminum or steel sheets. The derived process chains base on automated fiber placement, resin transfer molding, deep drawing, rotational molding and integral tube blow molding.

Systematic Process Chain Creation for CFRP Structures in Early Product Development Stages

2016 ◽  
Vol 1140 ◽  
pp. 328-334
Author(s):  
Matthias Behr ◽  
Carsten Schmidt
Keyword(s):  
Composite Structures ◽  
Production Costs ◽  
Preliminary Design ◽  
Process Chain ◽  
Whole Process ◽  
Reinforced Plastic ◽  
Building Process ◽  
Process Chains ◽  
Carbon Fibre Reinforced Plastic ◽  
Specific Sequences

A planning method is presented which allows to systematically building process chains based on a preliminary design of composite structures. The method utilises the specific sequences of procedural steps that occur in the production of carbon fibre reinforced plastic (CFRP) structures, to build sub process chains for each component of the structure. Process restrictions are considered to evaluate the suitability of different production processes. To obtain the whole process chain of the structure, different joining methods are applied in addition to combine the components and its sub process chains. The results of the presented method are used in an overarching development procedure to investigate resulting impacts on the solution. Possible impacts could be the production costs or the material characteristics.

Heuristic-Based Evaluation of Energy Flows in Press Hardening Process Chains

2011 ◽  
Vol 473 ◽  
pp. 816-823 ◽  
Author(s):  
Reimund Neugebauer ◽  
Frank Schieck ◽  
Angela Göschel ◽  
Julia Schönherr
Keyword(s):  
Qualitative Evaluation ◽  
Resource Efficiency ◽  
Process Chain ◽  
Press Hardening ◽  
Whole Process ◽  
Quantitative Calculation ◽  
Hardening Process ◽  
Energy Flows ◽  
Process Chains ◽  
The Press

Energy and resource efficiency is a pressing issue for technological markets in the 21st century. In the field of production technology the development of energy and resource efficient processes and process chains is of particular importance. In order to meet these needs sustainable methods and standards have to be developed. This paper presents a new procedure to calculate and evaluate the energy and resource efficiency of process chains. The method consists of 4 stages that proceed from the real world to the quantitative calculation and qualitative evaluation of material and energy flows. The method is explained and validated using press hardening process chains as an example. The procedure enables the user to systematically capture and structure the press hardening process chain and subsequently develop a comprehensive model of the whole process chain. As a result, it allows to calculate the energy requirements for each stage of the process chain, and later on the process chain as a whole. The intention of the developed procedure is to provide a tool to detect the most energy efficient variant from a range of possible process chains.

scholarly journals An Activity and Resource Advisory System for Manufacturing Process Chains Selection at the early Stage of Product Development

2013 ◽  
Vol 834-836 ◽  
pp. 1927-1931
Author(s):  
Jaya Suteja The ◽  
Prasad K.D.V. Yarlagadda ◽  
M. Azharul Karim ◽  
Cheng Yan
Keyword(s):  
Product Development ◽  
Production Process ◽  
Manufacturing Process ◽  
Early Stage ◽  
Process Chain ◽  
Advisory System ◽  
Process Chains ◽  
Manufacturing Process Chain

Designers need to consider both the functional and production process requirements at the early stage of product development. A variety of the research works found in the literature has been proposed to assist designers in selecting the most viable manufacturing process chain. However, they do not provide any assistance for designers to evaluate the processes according to the particular circumstances of their company. This paper describes a framework of an Activity and Resource Advisory System (ARAS) that generates advice about the required activities and the possible resources for various manufacturing process chains. The system provides more insight, more flexibility, and a more holistic and suited approach for designers to evaluate and then select the most viable manufacturing process chain at the early stage of product development.

scholarly journals WiFi Offloading Algorithm Based on Q-Learning and MADM in Heterogeneous Networks

2019 ◽  
Vol 2019 ◽  
pp. 1-12
Author(s):  
Lin Sun ◽  
Qi Zhu
Keyword(s):  
Heterogeneous Networks ◽  
User Satisfaction ◽  
Mobile User ◽  
User Cost ◽  
Analytic Hierarchy ◽  
Wifi Networks ◽  
Q Learning ◽  
Reward Function ◽  
Wifi Offloading ◽  
Topsis Technique

This paper proposes a WiFi offloading algorithm based on Q-learning and MADM (multiattribute decision making) in heterogeneous networks for a mobile user scenario where cellular networks and WiFi networks coexist. The Markov model is used to describe the changes of the network environment. Four attributes including user throughput, terminal power consumption, user cost, and communication delay are considered to define the user satisfaction function reflecting QoS (Quality of Service), and Q-learning is used to optimize it. Through AHP (Analytic Hierarchy Process) and TOPSIS (Technique for Order Preference by Similarity to an Ideal Solution) in MADM, the intrinsic connection between each attribute and the reward function is obtained. The user uses Q-learning to make offloading decisions based on current network conditions and their own offloading history, ultimately maximizing their satisfaction. The simulation results show that the user satisfaction of the proposed algorithm is better than the traditional WiFi offloading algorithm.

scholarly journals A deep Q-learning portfolio management framework for the cryptocurrency market

2020 ◽  
Vol 32 (23) ◽  
pp. 17229-17244
Author(s):  
Giorgio Lucarelli ◽  
Matteo Borrotti
Keyword(s):  
Portfolio Optimization ◽  
Portfolio Management ◽  
Dynamic Decision Making ◽  
Financial Assets ◽  
Management Framework ◽  
Q Learning ◽  
Reward Function ◽  
Dynamic Portfolio ◽  
Definition Of ◽  
Dynamic Portfolio Optimization

AbstractDeep reinforcement learning is gaining popularity in many different fields. An interesting sector is related to the definition of dynamic decision-making systems. A possible example is dynamic portfolio optimization, where an agent has to continuously reallocate an amount of fund into a number of different financial assets with the final goal of maximizing return and minimizing risk. In this work, a novel deep Q-learning portfolio management framework is proposed. The framework is composed by two elements: a set of local agents that learn assets behaviours and a global agent that describes the global reward function. The framework is tested on a crypto portfolio composed by four cryptocurrencies. Based on our results, the deep reinforcement portfolio management framework has proven to be a promising approach for dynamic portfolio optimization.

Microfabrication of ceramics by rapid prototyping process chains

2003 ◽  
Vol 217 (1) ◽  
pp. 41-51 ◽  
Author(s):  
R Knitter ◽  
W Bauer ◽  
D Göhring
Keyword(s):  
Rapid Prototyping ◽  
Process Chain ◽  
Moulding Process ◽  
Pressure Injection ◽  
Functional Models ◽  
Process Chains ◽  
Ceramic Components ◽  
High Flexibility ◽  
Very High

Most shaping processes for ceramics are based on a powder technological moulding process using a negative mould and subsequent thermal compaction. Especially for prototypes and small-lot series of microcomponents, the outlay for moulds is the major costing factor. Therefore the use of rapid prototyping (RP) processes can decisively reduce the costs and time in product development of ceramic microcomponents. By combining the high resolution of, for example, stereolithography as an inexpensive and fast supply for master models with the high flexibility of low-pressure injection moulding, a rapid prototyping process chain (RPPC) has been established for the fabrication of micropatterned ceramic components as functional models or pre-production lots. This RPPC proved to have a very high moulding precision and accuracy in the submillimetre range, but also enables the fabrication of components with outer dimensions of several centimetres. Different RP techniques were investigated with regard to their suitability to be used as master models in the replication chain. The quality of the master models turned out to be of decisive significance for the quality and reproducibility of the ceramic mouldings.

Methods for the Control of Uncertainty in Multilevel Process Chains Using the Example of Drilling/Reaming

2011 ◽  
Vol 104 ◽  
pp. 103-113 ◽  
Author(s):  
Michael Haydn ◽  
Thomas Hauer ◽  
Eberhard Abele
Keyword(s):  
Systematic Approach ◽  
Simulation Models ◽  
Production Costs ◽  
Material Strength ◽  
Process Chain ◽  
Production Processes ◽  
Process Chains ◽  
Key Aspects ◽  
And Control ◽  
Previous Step

Uncertainty during production processes has an important influence on the product quality as well as production costs. For multilevel process chains with serially connected processes, additional uncertainty can be caused by the previous step. The manufacturing of precision holes by drilling and reaming is an important multilevel process chain. The interactions between machine, tool and pre-drilled hole cause process errors during the quality determinant final reaming process. In this paper, a systematic approach for the identification and control of uncertainty during the reaming process is presented. Thus, the influence of key aspects like skewness of pre-drilled hole or the influences of material strength gradients are analyzed. Further, simulation models for the consideration of these uncertainties are presented.

Sign in / Sign up

Export Citation Format

Share Document