Reducing Long-Run Average Planned Maintenance Cost Using Markov Decision Modelling Based on Shifting Paradigm and Penalty Model

2022 ◽  
pp. 277-294
Author(s):  
Gedefaye Achamu Meretie ◽  
Eshetie Berhan Atanew ◽  
Sisay Geremaw Gebeyehu
Keyword(s):  
Maintenance Cost ◽  
Decision Modelling ◽  
Long Run ◽  
Markov Decision

Optimizing the re-profiling policy regarding metropolitan train wheels based on a semi-Markov decision process

2017 ◽  
Vol 231 (5) ◽  
pp. 495-507 ◽  
Author(s):  
Zengqiang Jiang ◽  
Dragan Banjevic ◽  
Mingcheng E ◽  
Bing Li
Keyword(s):  
Markov Decision Process ◽  
Condition Monitoring ◽  
Decision Process ◽  
Maintenance Cost ◽  
Iteration Algorithm ◽  
Long Run ◽  
Flange Thickness ◽  
Markov Decision ◽  
Dimensional State Space ◽  
Policy Iteration Algorithm

In this article, we present a maintenance model for metropolitan train wheels subjected to diameter or flange thickness overruns that includes condition monitoring with periodic inspection. We present a dynamic ([Formula: see text], [Formula: see text]) policy based on condition monitoring information, where [Formula: see text] is the wheel flange thickness threshold that triggers preventive re-profiling and [Formula: see text] is the recovery value for the wheel flange thickness after preventive re-profiling. The problem is modelled as a semi-Markov decision process that considers wear in terms of the diameter and flange thickness simultaneously. The problem is formulated in a two-dimensional state space; this space is defined as a combination of the diameter state and the flange thickness state. The model also considers imperfect wheel maintenance. The model’s objective is to minimize the maintenance cost per unit time that is expected in the long run. We apply a policy-iteration algorithm as the computational approach to determine the optimal re-profiling policy and use an example to demonstrate the method’s effectiveness.

Availability analysis and maintenance optimization for multiple failure mode systems considering imperfect repair

2021 ◽  
pp. 1748006X2110127
Author(s):  
Qingan Qiu ◽  
Baoliang Liu ◽  
Cong Lin ◽  
Jingjing Wang
Keyword(s):  
Failure Modes ◽  
Maintenance Cost ◽  
Imperfect Maintenance ◽  
Maintenance Policy ◽  
Cost Rate ◽  
Availability Analysis ◽  
Long Run ◽  
Optimal Maintenance ◽  
Maintenance Policies ◽  
Periodic Inspections

This paper studies the availability and optimal maintenance policies for systems subject to competing failure modes under continuous and periodic inspections. The repair time distribution and maintenance cost are both dependent on the failure modes. We investigate the instantaneous availability and the steady state availability of the system maintained through several imperfect repairs before a replacement is allowed. Analytical expressions for system availability under continuous and periodic inspections are derived respectively. The availability models are then utilized to obtain the optimal inspection and imperfect maintenance policy that minimizes the average long-run cost rate. A numerical example for Remote Power Feeding System is presented to demonstrate the application of the developed approach.

scholarly journals Markov Decision Processes with Multiple Long-run Average Objectives

2014 ◽  
Vol 10 (1) ◽  
Author(s):  
Tomáš Brázdil ◽  
Václav Brožek ◽  
Krishnendu Chatterjee ◽  
Vojtěch Forejt ◽  
Antonín Kučera
Keyword(s):  
Markov Decision Processes ◽  
Decision Processes ◽  
Long Run ◽  
Markov Decision

scholarly journals Markov decision processes with quasi-hyperbolic discounting

2020 ◽  
Author(s):  
Anna Jaśkiewicz ◽  
Andrzej S. Nowak
Keyword(s):  
Markov Decision Processes ◽  
Dynamic Game ◽  
Hyperbolic Discounting ◽  
Decision Processes ◽  
State Spaces ◽  
Long Run ◽  
Markov Perfect ◽  
Markov Decision ◽  
Time Inconsistent ◽  
Perfect Equilibria

AbstractWe study Markov decision processes with Borel state spaces under quasi-hyperbolic discounting. This type of discounting nicely models human behaviour, which is time-inconsistent in the long run. The decision maker has preferences changing in time. Therefore, the standard approach based on the Bellman optimality principle fails. Within a dynamic game-theoretic framework, we prove the existence of randomised stationary Markov perfect equilibria for a large class of Markov decision processes with transitions having a density function. We also show that randomisation can be restricted to two actions in every state of the process. Moreover, we prove that under some conditions, this equilibrium can be replaced by a deterministic one. For models with countable state spaces, we establish the existence of deterministic Markov perfect equilibria. Many examples are given to illustrate our results, including a portfolio selection model with quasi-hyperbolic discounting.

Optimal control of a truncated general immigration process through total catastrophes

1999 ◽  
Vol 36 (02) ◽  
pp. 461-472
Author(s):  
E. G. Kyriakidis
Keyword(s):  
Decision Model ◽  
Optimality Criterion ◽  
Control Limit ◽  
Pest Population ◽  
Decision Algorithm ◽  
Long Run ◽  
Immigration Process ◽  
Markov Decision ◽  
Markov Decision Model ◽  
Necessary And Sufficient

A Markov decision model is considered for the control of a truncated general immigration process, which represents a pest population, by the introduction of total catastrophes. The optimality criterion is that of minimizing the expected long-run average cost per unit time. Firstly, a necessary and sufficient condition is found under which the policy of never controlling is optimal. If this condition fails, a parametric analysis, in which a fictitious parameter is varied over the entire real line, is used to establish the optimality of a control-limit policy. Furthermore, an efficient Markov decision algorithm operating on the class of control-limit policies is developed for the computation of the optimal policy.

scholarly journals Nonstationary value iteration in controlled Markov chains with risk-sensitive average criterion

2005 ◽  
Vol 42 (4) ◽  
pp. 905-918 ◽  
Author(s):  
Rolando Cavazos-Cadena ◽  
Raúl Montes-De-Oca
Keyword(s):  
Iteration Algorithm ◽  
Value Iteration ◽  
Stationary Policy ◽  
Long Run ◽  
Risk Sensitive ◽  
Finite State ◽  
Markov Decision ◽  
Optimal Stationary Policy ◽  
Compact Action Sets ◽  
Nonstationary Value Iteration

This work concerns Markov decision chains with finite state spaces and compact action sets. The performance index is the long-run risk-sensitive average cost criterion, and it is assumed that, under each stationary policy, the state space is a communicating class and that the cost function and the transition law depend continuously on the action. These latter data are not directly available to the decision-maker, but convergent approximations are known or are more easily computed. In this context, the nonstationary value iteration algorithm is used to approximate the solution of the optimality equation, and to obtain a nearly optimal stationary policy.

scholarly journals Core Nonemptiness of Stratified Pooling Games: A Structured Markov Decision Process Approach

2020 ◽  
Vol 45 (4) ◽  
pp. 1445-1465
Author(s):  
Loe Schlicher ◽  
Marco Slikker ◽  
Willem van Jaarsveld ◽  
Geert-Jan van Houtum
Keyword(s):  
Markov Decision Process ◽  
Cooperative Game ◽  
Decision Process ◽  
Service Providers ◽  
Spare Parts ◽  
High Tech ◽  
Long Run ◽  
Markov Decision ◽  
Coalitional Values ◽  
The Individual

We study several service providers that keep spare parts in stock to protect for downtime of their high-tech machines and that face different downtime costs per stockout. Service providers can cooperate by forming a joint spare parts pool, and we study the allocation of the joint costs to the individual service providers by studying an associated cooperative game. In extant literature, the joint spare parts pool is typically controlled by a suboptimal full-pooling policy. A full-pooling policy may lead to an empty core of the associated cooperative game, and we show this result in our setting as well. We then focus on situations where service providers apply an optimal policy: a stratification that determines, depending on the real-time on-hand inventory, which service providers may take parts from the pool. We formulate the associated stratified pooling game by defining each coalitional value in terms of the minimal long-run average costs of a Markov decision process. We present a proof demonstrating that stratified pooling games always have a nonempty core. This five-step proof is of interest in itself, because it may be more generally applicable for other cooperative games where coalitional values can be defined in terms of Markov decision processes.

Two competing queues with linear costs: the μc-rule is often optimal

1984 ◽  
Vol 16 (1) ◽  
pp. 8-8
Author(s):  
J. S. Baras ◽  
A. J. Dorsey ◽  
A. M. Makowski
Keyword(s):  
State Space ◽  
Random Sequence ◽  
State Space Model ◽  
Sample Path ◽  
Single Server ◽  
Space Model ◽  
Long Run ◽  
Markov Decision ◽  
Cost Parameters ◽  
Discounted Criterion

A state-space model is presented for a queueing system where two classes of customer compete in discrete-time for the service attention of a single server with infinite buffer capacity. The arrivals are modelled by an independent identically distributed random sequence of a general type while the service completions are generated by independent Bernoulli streams; the allocation of service attention is governed by feedback policies which are based on past decisions and buffer content histories. The cost of operation per unit time is a linear function of the queue sizes. Under the model assumptions, a fixed prioritization scheme, known as the μc -rule, is shown to be optimal when the expected long-run average criterion and the expected discounted criterion, over both finite and infinite horizons, are used. This static prioritization of the two classes of customers is done solely on the basis of service and cost parameters. The analysis is based on the dynamic programming methodology for Markov decision processes and takes advantage of the sample-path properties of the adopted state-space model.

Reliability and maintenance modeling for competing failures with intermission considered

2019 ◽  
Vol 233 (5) ◽  
pp. 898-907
Author(s):  
Jingyi Liu ◽  
Yugang Zhang ◽  
Bifeng Song
Keyword(s):  
System Reliability ◽  
Rest Period ◽  
Maintenance Cost ◽  
Reliability Model ◽  
Maintenance Policy ◽  
General Applicability ◽  
Industrial Systems ◽  
Soft Failure ◽  
Long Run ◽  
Degradation Failure

There are many industrial systems experiencing multiple dependent competing failure processes, in detail degradation failure (soft failure) and catastrophic failure (hard failure). Earlier research studied failure behaviors and system reliability during operational period, but did not consider the intermission period. Some industrial systems are not always operating continuously while with intermissions or rest period. The degradation and random shock processes are different between operating period and intermissions, which caused it more challenging and complicated to establish reliability model. In this article, a new reliability model for multiple dependent competing failure processes is developed with intermission considered. The system reliability can be analyzed based on the proposed model more practically. Besides, a preventive replacement maintenance policy is studied by minimizing the average long-run maintenance cost with intermission periods considered. Finally, the availability and general applicability of presented model are demonstrated by a case in different parameter settings.

Sign in / Sign up

Export Citation Format

Share Document