scholarly journals Analysis of Risks and Costs in Intruder Detection With Markov Decision Processes

2021 ◽  
Author(s):  
Jorma Jormakka ◽  
Sourangshu Ghosh
Keyword(s):  
Markov Decision Processes ◽  
False Positives ◽  
Decision Processes ◽  
Defence Mechanisms ◽  
False Negatives ◽  
Alternative Strategies ◽  
Intruder Detection ◽  
Markov Decision ◽  
The Difference

Let us assume that defence mechanisms are so strong that the average outcome of a hacking attack is unsuccessful. How to calculate the costs arising from false positives and false negatives in intruder detection? Is it better for the hacker to make fewer but more effective attacks rather than several but less effective attacks? How to calculate the difference between these alternative strategies?

scholarly journals First Passage Optimality and Variance Minimisation of Markov Decision Processes with Varying Discount Factors

2015 ◽  
Vol 52 (2) ◽  
pp. 441-456 ◽  
Author(s):  
Xiao Wu ◽  
Xianping Guo
Keyword(s):  
Markov Decision Processes ◽  
Discrete Time ◽  
Infinite Horizon ◽  
Decision Processes ◽  
Previous Literature ◽  
First Passage ◽  
Discount Factors ◽  
Markov Decision ◽  
The Difference ◽  
Minimisation Problem

This paper deals with the first passage optimality and variance minimisation problems of discrete-time Markov decision processes (MDPs) with varying discount factors and unbounded rewards/costs. First, under suitable conditions slightly weaker than those in the previous literature on the standard (infinite horizon) discounted MDPs, we establish the existence and characterisation of the first passage expected-optimal stationary policies. Second, to further distinguish the expected-optimal stationary policies, we introduce the variance minimisation problem, prove that it is equivalent to a new first passage optimality problem of MDPs, and, thus, show the existence of a variance-optimal policy that minimises the variance over the set of all first passage expected-optimal stationary policies. Finally, we use a computable example to illustrate our main results and also to show the difference between the first passage optimality here and the standard discount optimality of MDPs in the previous literature.

First Passage Optimality and Variance Minimisation of Markov Decision Processes with Varying Discount Factors

2015 ◽  
Vol 52 (02) ◽  
pp. 441-456 ◽  
Author(s):  
Xiao Wu ◽  
Xianping Guo
Keyword(s):  
Markov Decision Processes ◽  
Discrete Time ◽  
Infinite Horizon ◽  
Decision Processes ◽  
Previous Literature ◽  
First Passage ◽  
Discount Factors ◽  
Markov Decision ◽  
The Difference ◽  
Minimisation Problem

This paper deals with the first passage optimality and variance minimisation problems of discrete-time Markov decision processes (MDPs) with varying discount factors and unbounded rewards/costs. First, under suitable conditions slightly weaker than those in the previous literature on the standard (infinite horizon) discounted MDPs, we establish the existence and characterisation of the first passage expected-optimal stationary policies. Second, to further distinguish the expected-optimal stationary policies, we introduce the variance minimisation problem, prove that it is equivalent to a new first passage optimality problem of MDPs, and, thus, show the existence of a variance-optimal policy that minimises the variance over the set of all first passage expected-optimal stationary policies. Finally, we use a computable example to illustrate our main results and also to show the difference between the first passage optimality here and the standard discount optimality of MDPs in the previous literature.

Extreme-point solutions in Markov decision processes

1983 ◽  
Vol 20 (04) ◽  
pp. 835-842
Author(s):  
David Assaf
Keyword(s):  
Convex Function ◽  
Extreme Point ◽  
Markov Decision Processes ◽  
Convex Functions ◽  
Sufficient Conditions ◽  
Decision Processes ◽  
Markov Decision ◽  
Full Solution

The paper presents sufficient conditions for certain functions to be convex. Functions of this type often appear in Markov decision processes, where their maximum is the solution of the problem. Since a convex function takes its maximum at an extreme point, the conditions may greatly simplify a problem. In some cases a full solution may be obtained after the reduction is made. Some illustrative examples are discussed.

Sign in / Sign up

Export Citation Format

Share Document