Stochastic mixed model sequencing with multiple stations using reinforcement learning and probability quantiles

In this study, we propose a reinforcement learning (RL) approach for minimizing the number of work overload situations in the mixed model sequencing (MMS) problem with stochastic processing times. The learning environment simulates stochastic processing times and penalizes work overloads with negative rewards. To account for the stochastic component of the problem, we implement a state representation that specifies whether work overloads will occur if the processing times are equal to their respective 25%, 50%, and 75% probability quantiles. Thereby, the RL agent is guided toward minimizing the number of overload situations while being provided with statistical information about how fluctuations in processing times affect the solution quality. To the best of our knowledge, this study is the first to consider the stochastic problem variation with a minimization of overload situations.

Diffusion Approximation for Fair Resource Control—Interchange of Limits Under a Moment Condition

In a prior study [Ye HQ, Yao DD (2016) Diffusion limit of fair resource control–Stationary and interchange of limits. Math. Oper. Res. 41(4):1161–1207.] focusing on a class of stochastic processing network with fair resource control, we justified the diffusion approximation (in the context of the interchange of limits) provided that the pth moment of the workloads are bounded. To this end, we introduced the so-called bounded workload condition, which requires the workload process be bounded by a free process plus the initial workload. This condition is for a derived process, the workload, as opposed to primitives such as arrival processes and service requirements; as such, it could be difficult to verify. In this paper, we establish the interchange of limits under a moment condition of suitable order on the primitives directly: the required order is [Formula: see text] on the moments of the primitive processes so as to bound the pth moment of the workload. This moment condition is trivial to verify, and indeed automatically holds in networks where the primitives have moments of all orders, for instance, renewal arrivals with phase-type interarrival times and independent and identically distributed phase-type service times.

Information and Memory in Dynamic Resource Allocation

We propose a general framework, dubbed stochastic processing under imperfect information (SPII), to study the impact of information constraints and memories on dynamic resource allocation. The framework involves a stochastic processing network (SPN) scheduling problem in which the scheduler may access the system state only through a noisy channel, and resource allocation decisions must be carried out through the interaction between an encoding policy (that observes the state) and an allocation policy (that chooses the allocation). Applications in the management of large-scale data centers and human-in-the-loop service systems are among our chief motivations. We quantify the degree to which information constraints reduce the size of the capacity region in general SPNs and how such reduction depends on the amount of memories available to the encoding and allocation policies. Using a novel metric, capacity factor, our main theorem characterizes the reduction in capacity region (under “optimal” policies) for all nondegenerate channels and across almost all combinations of memory sizes. Notably, the theorem demonstrates, in substantial generality, that (1) the presence of a noisy channel always reduces capacity, (2) more memory for the allocation policy always improves capacity, and (3) more memory for the encoding policy has little to no effect on capacity. Finally, all of our positive (achievability) results are established through constructive, implementable policies.

JOB-SHOP SCHEDULING OPTIMIZATION WITH STOCHASTIC PROCESSING TIMES

In this study, a job shop scheduling optimization model under risk has been developed to minimize the make span. This model has been built using Microsoft Excel spreadsheets and solved using @Risk solver. A set of experiments have been also conducted to examine the accuracy of the model and its effectiveness has been proven.