scholarly journals Optimal Search for a Moving Target

1995 ◽  
Vol 9 (2) ◽  
pp. 159-182 ◽  
Author(s):  
I. M. MacPhee ◽  
B. P. Jordan
Keyword(s):  
Markov Chain ◽  
Optimal Policy ◽  
Average Cost ◽  
Large Range ◽  
Moving Target ◽  
Optimal Search ◽  
Threshold Probability

Consider the problem of searching for a leprechaun that moves randomly between two sites. The movement is modelled with a two-state Markov chain. One of the sites is searched at each time t = 1,2,…, until the leprechaun is found. Associated with each search of site i is an overlook probability αi and a cost Ci Our aim is to determine the policy that will find the leprechaun with the minimal average cost. Let p denote the probability that the leprechaun is at site 1. Ross conjectured that an optimal policy can be defined in terms of a threshold probability P* such that site 1 is searched if and only if p ≥ P*. We show this conjecture to be correct (i) when α1 = α2 and C1 = C2, (ii) for general Ci when the overlook probabilities α, are small, and (iii) for general αi and Ci for a large range of transition laws for the movement. We also derive some properties of the optimal policy for the problem on n sites in the no-overlook case and for the case where each site has the same αi, and Ci.

scholarly journals Search for a moving target in a competitive environment

2021 ◽  
Author(s):  
Benoit Duvocelle ◽  
János Flesch ◽  
Hui Min Shi ◽  
Dries Vermeulen
Keyword(s):  
Markov Chain ◽  
Discrete Time ◽  
Competitive Environment ◽  
Moving Target ◽  
Time Varying ◽  
Greedy Strategy ◽  
Time Dynamic ◽  
Dynamic Search ◽  
Subgame Perfect Equilibria ◽  
Perfect Equilibria

AbstractWe consider a discrete-time dynamic search game in which a number of players compete to find an invisible object that is moving according to a time-varying Markov chain. We examine the subgame perfect equilibria of these games. The main result of the paper is that the set of subgame perfect equilibria is exactly the set of greedy strategy profiles, i.e. those strategy profiles in which the players always choose an action that maximizes their probability of immediately finding the object. We discuss various variations and extensions of the model.

scholarly journals Optimal Search and Discovery

2021 ◽  
Author(s):  
Rafael P. Greminger
Keyword(s):  
Optimal Policy ◽  
Purchase Decision ◽  
Search Problem ◽  
Directed Search ◽  
Optimal Search

This paper studies a search problem in which a consumer is initially aware of only a few products. At every point in time, the consumer then decides between searching among alternatives of which he is already aware and discovering more products. I show that the optimal policy for this search and discovery problem is fully characterized by tractable reservation values. Moreover, I prove that a predetermined index fully specifies the purchase decision of a consumer following the optimal search policy. Finally, a comparison highlights differences to classical random and directed search. This paper was accepted by Dmitri Kuksov, marketing.

scholarly journals Limiting Behavior of the Target-Dependent Stochastic Sequential Assignment Problem

2014 ◽  
Vol 51 (4) ◽  
pp. 943-953 ◽  
Author(s):  
Golshid Baharian ◽  
Sheldon H. Jacobson
Keyword(s):  
Distribution Function ◽  
Markov Chain ◽  
Assignment Problem ◽  
Sequential Assignment ◽  
Threshold Probability ◽  
Limiting Behavior ◽  
Target Value ◽  
Long Run ◽  
Stochastic Parameters ◽  
Optimal Policies

The stochastic sequential assignment problem assigns distinct workers to sequentially arriving tasks with stochastic parameters. In this paper the assignments are performed so as to minimize the threshold probability, which is the probability of the long-run reward per task failing to achieve a target value (threshold). As the number of tasks approaches infinity, the problem is studied for independent and identically distributed (i.i.d.) tasks with a known distribution function and also for tasks that are derived from r distinct unobservable distributions (governed by a Markov chain). Stationary optimal policies are presented, which simultaneously minimize the threshold probability and achieve the optimal long-run expected reward per task.

On the Kesten–Spitzer algorithm for minimal hitting time in controlled Markov chains

1980 ◽  
Vol 17 (03) ◽  
pp. 716-725
Author(s):  
Manish C. Bhattacharjee ◽  
Sujit K. Basu
Keyword(s):  
Markov Chain ◽  
Optimal Policy ◽  
Similar Condition ◽  
Control Policy ◽  
Hitting Time ◽  
Sufficient Condition ◽  
Expected Time ◽  
Fixed State ◽  
Necessary And Sufficient ◽  
Modified Algorithm

For a Markov chain with optional transitions, except for those to an arbitrary fixed state accessible from all others, Kesten and Spitzer proved the existence of a control policy which minimized the expected time to reach the fixed state and for constructing an optimal policy, proposed an algorithm which works in certain cases. For the algorithm to work they gave a sufficient condition which breaks down if there are countably many states and the minimal hitting time is bounded. We propose a modified algorithm which is shown to work under a weaker sufficient condition. In the bounded case with countably many states, the proposed sufficient condition is not necessary but a similar condition is. In the unbounded case as well as when the state space is finite, the proposed condition is shown to be both necessary and sufficient for the original Kesten–Spitzer algorithm to work. A new iterative algorithm which can be used in all cases is given.

Optimal pest control through catastrophes

1989 ◽  
Vol 26 (04) ◽  
pp. 873-879 ◽  
Author(s):  
E.G. Kyriakidis ◽  
Andris Abakuks
Keyword(s):  
Population Size ◽  
Pest Control ◽  
Optimal Policy ◽  
Average Cost ◽  
Optimality Criterion ◽  
Critical Value ◽  
Restricted Class ◽  
Pest Population ◽  
Birth Process

This paper is concerned with the problem of controlling a simple immigration–birth process, which represents a pest population, by the introduction of catastrophes which, when they occur, reduce the population size to zero. The optimality criterion is that of minimising the long-term average cost per unit time of the process. Firstly, an optimal policy is found within a restricted class of stationary policies, which introduce catastrophes if and only if the population size is greater than or equal to some critical value x. The optimality of this policy within the wider class of all stationary policies is then verified by applying the general results of Bather (1976).

Optimal control of a finite dam: average-cost case

1985 ◽  
Vol 22 (02) ◽  
pp. 480-484 ◽  
Author(s):  
Lam Yeh
Keyword(s):  
Optimal Control ◽  
Wiener Process ◽  
Release Rate ◽  
Optimal Policy ◽  
Average Cost ◽  
Output Rate ◽  
Penalty Cost ◽  
Long Run ◽  
Negative Drift ◽  
Finite Dam

We consider the problem of minimizing the long-run average cost per unit time of operating a finite dam in the class of the policies of the following type. Assume that the dam is initially empty, the release rate is kept at 0 until the dam storage increases to λ, and as soon as this occurs, water is released at rate M, then the output rate is kept at M as long as the dam storage is more than τ and it must be decreased to 0 if the dam storage becomes τ. We assume that the input of water into the finite dam is a Wiener process with non-negative drift μ and variance parameter σ 2. There is a cost in increasing the output rate from 0 to M as well as in decreasing the rate from M to 0 and whenever the dam storage is below level a, there is a penalty cost per unit time depending on the level. A reward is given for each unit of water released. In this paper, the long-run average cost per unit time is determined, and therefore the optimal policy can be found numerically.

Optimal search for a randomly moving object

1986 ◽  
Vol 23 (3) ◽  
pp. 708-717 ◽  
Author(s):  
R. R. Weber
Keyword(s):  
Markov Process ◽  
Optimal Policy ◽  
Moving Object ◽  
The Other ◽  
Optimal Search ◽  
Expected Cost

It is desired to minimize the expected cost of finding an object which moves back and forth between two locations according to an unobservable Markov process. When the object is in location i (i = 1, 2) it resides there for a time which is exponentially distributed with parameter λ1 and then moves to the other location. The location of the object is not known and at each instant until it is found exactly one of the two locations must be searched. Searching location i for time δ costs ciδ and conditional on the object being in location i there is a probability αiδ + o(δ) that this search will find it. The probability that the object starts in location 1 is known to bé p1(0). The location to be searched at time t is to be chosen on the basis of the value of p1(t), the probability that the object is in location 1, given that it has not yet been discovered. We prove that there exists a threshold Π such that the optimal policy may be described as: search location 1 if and only if the probability that the object is in location 1 is greater than Π. Expressions for the threshold Π are given in terms of the parameters of the model.

scholarly journals On the control of a truncated general immigration process through the introduction of a predator

2006 ◽  
Vol 2006 ◽  
pp. 1-12
Author(s):  
E. G. Kyriakidis
Keyword(s):  
Real Line ◽  
Optimal Policy ◽  
Average Cost ◽  
Control Limit ◽  
Decision Algorithm ◽  
Mild Conditions ◽  
Entire Real Line ◽  
Immigration Process ◽  
Markov Decision

This paper is concerned with the problem of controlling a truncated general immigration process, which represents a population of harmful individuals, by the introduction of a predator. If the parameters of the model satisfy some mild conditions, the existence of a control-limit policy that is average-cost optimal is proved. The proof is based on the uniformization technique and on the variation of a fictitious parameter over the entire real line. Furthermore, an efficient Markov decision algorithm is developed that generates a sequence of improving control-limit policies converging to the optimal policy.

The control of a finite dam

1982 ◽  
Vol 19 (4) ◽  
pp. 815-825 ◽  
Author(s):  
F. A. Attia ◽  
P. J. Brockwell
Keyword(s):  
Markov Chain ◽  
Wiener Process ◽  
Average Cost ◽  
Type B ◽  
Input Process ◽  
Long Run ◽  
The Cost ◽  
Finite Dam

The long-run average cost per unit time of operating a finite dam controlled by a PlM policy (Faddy (1974), Zuckerman (1977)) is determined when the cumulative input process is (a) a Wiener process with drift and (b) the integral of a Markov chain. It is shown how the cost for (a) can be obtained as the limit of the costs associated with a sequence of input processes of the type (b).

Sign in / Sign up

Export Citation Format

Share Document