scholarly journals An Optimal Flow Admission and Routing Control Policy for Resource Constrained Networks

Sensors ◽  
2020 ◽  
Vol 20 (22) ◽  
pp. 6566
Author(s):  
Essia Hamouda
Keyword(s):  
Optimal Policy ◽  
Control Policy ◽  
Wireless Devices ◽  
Markov Decision Problem ◽  
Large Size ◽  
Resource Limited ◽  
State Dependent ◽  
Markov Decision ◽  
Routing Control ◽  
Network Device

Overloaded network devices are becoming an increasing problem especially in resource limited networks with the continuous and rapid increase of wireless devices and the huge volume of data generated. Admission and routing control policy at a network device can be used to balance the goals of maximizing throughput and ensuring sufficient resources for high priority flows. In this paper we formulate the admission and routing control problem of two types of flows where one has a higher priority than the other as a Markov decision problem. We characterize the optimal admission and routing policy, and show that it is a state-dependent threshold type policy. Furthermore, we conduct extensive numerical experiments to gain more insight into the behavior of the optimal policy under different systems’ parameters. While dynamic programming can be used to solve such problems, the large size of the state space makes it untractable and too resource intensive to run on wireless devices. Therefore, we propose a fast heuristic that exploits the structure of the optimal policy. We empirically show that the heuristic performs very well with an average reward deviation of 1.4% from the optimal while being orders of magnitude faster than the optimal policy. We further generalize the heuristic for the general case of a system with n (n>2) types of flows.

scholarly journals Fault-Tolerant Control of a Distributed Database System

2008 ◽  
Vol 2008 ◽  
pp. 1-13 ◽  
Author(s):  
N. Eva Wu ◽  
Matthew C. Ruschmann ◽  
Mark H. Linderman
Keyword(s):  
Fault Tolerant ◽  
Control Policy ◽  
Distributed Database ◽  
Database System ◽  
Data Sets ◽  
Single Server ◽  
Markov Decision Problem ◽  
Redundant Data ◽  
Markov Decision ◽  
Distributed Database System

Optimal state information-based control policy for a distributed database system subject to server failures is considered. Fault-tolerance is made possible by the partitioned architecture of the system and data redundancy therein. Control actions include restoration of lost data sets in a single server using redundant data sets in the remaining servers, routing of queries to intact servers, or overhaul of the entire system for renewal. Control policies are determined by solving Markov decision problems with cost criteria that penalize system unavailability and slow query response. Steady-state system availability and expected query response time of the controlled database are evaluated with the Markov model of the database. Robustness is addressed by introducing additional states into the database model to account for control action delays and decision errors. A robust control policy is solved for the Markov decision problem described by the augmented state model.

Dynamic Pricing and Routing for Same-Day Delivery

2020 ◽  
Vol 54 (4) ◽  
pp. 1016-1033 ◽  
Author(s):  
Marlin W. Ulmer
Keyword(s):  
Dynamic Pricing ◽  
Function Approximation ◽  
Computational Study ◽  
Routing Problem ◽  
Value Function Approximation ◽  
Routing Policy ◽  
State Dependent ◽  
Markov Decision ◽  
Fixed Prices ◽  
Number Of Customers

An increasing number of e-commerce retailers offers same-day delivery. To deliver the ordered goods, providers dynamically dispatch a fleet of vehicles transporting the goods from the warehouse to the customers. In many cases, retailers offer different delivery deadline options, from four-hour delivery up to next-hour delivery. Due to the deadlines, vehicles often only deliver a few orders per trip. The overall number of served orders within the delivery horizon is small and the revenue low. As a result, many companies currently struggle to conduct same-day delivery cost-efficiently. In this paper, we show how dynamic pricing is able to substantially increase both revenue and the number of customers we are able to serve the same day. To this end, we present an anticipatory pricing and routing policy (APRP) method that incentivizes customers to select delivery deadline options efficiently for the fleet to fulfill. This maintains the fleet’s flexibility to serve more future orders. We model the respective pricing and routing problem as a Markov decision process (MDP). To apply APRP, the state-dependent opportunity costs per customer and option are required. To this end, we use a guided offline value function approximation (VFA) based on state space aggregation. The VFA approximates the opportunity cost for every state and delivery option with respect to the fleet’s flexibility. As an offline method, APRP is able to determine suitable prices instantly when a customer orders. In an extensive computational study, we compare APRP with a policy based on fixed prices and with conventional temporal and geographical pricing policies. APRP outperforms the benchmark policies significantly, leading to both a higher revenue and more customers served the same day.

A Lightweight Authentication and Encryption Protocol for Secure Communications Between Resource-Limited Devices Without Hardware Modification

2018 ◽  
pp. 1-46
Author(s):  
Piotr Ksiazak ◽  
William Farrelly ◽  
Kevin Curran
Keyword(s):  
Ubiquitous Computing ◽  
Secure Communications ◽  
Wireless Devices ◽  
Authentication Protocols ◽  
It Industry ◽  
Processing Power ◽  
Resource Limited ◽  
Encryption Algorithms ◽  
Lightweight Authentication ◽  
Lightweight Encryption

In this chapter, the authors examine the theoretical context for the security of wireless communication between ubiquitous computing devices and present an implementation that addresses this need. The number of resource-limited wireless devices utilized in many areas of the IT industry is growing rapidly. Some of the applications of these devices pose real security threats that can be addressed using authentication and cryptography. Many of the available authentication and encryption software solutions are predicated on the availability of ample processing power and memory. These demands cannot be met by most ubiquitous computing devices; thus, there is a need to apply lightweight cryptography primitives and lightweight authentication protocols that meet these demands in any application of security to devices with limited resources. The analysis of the lightweight solutions is divided into lightweight authentication protocols and lightweight encryption algorithms. The authors present a prototype running on the nRF9E5 microcontroller that provides necessary authentication and encryption on resource-limited devices.

On the Kesten–Spitzer algorithm for minimal hitting time in controlled Markov chains

1980 ◽  
Vol 17 (03) ◽  
pp. 716-725
Author(s):  
Manish C. Bhattacharjee ◽  
Sujit K. Basu
Keyword(s):  
Markov Chain ◽  
Optimal Policy ◽  
Similar Condition ◽  
Control Policy ◽  
Hitting Time ◽  
Sufficient Condition ◽  
Expected Time ◽  
Fixed State ◽  
Necessary And Sufficient ◽  
Modified Algorithm

For a Markov chain with optional transitions, except for those to an arbitrary fixed state accessible from all others, Kesten and Spitzer proved the existence of a control policy which minimized the expected time to reach the fixed state and for constructing an optimal policy, proposed an algorithm which works in certain cases. For the algorithm to work they gave a sufficient condition which breaks down if there are countably many states and the minimal hitting time is bounded. We propose a modified algorithm which is shown to work under a weaker sufficient condition. In the bounded case with countably many states, the proposed sufficient condition is not necessary but a similar condition is. In the unbounded case as well as when the state space is finite, the proposed condition is shown to be both necessary and sufficient for the original Kesten–Spitzer algorithm to work. A new iterative algorithm which can be used in all cases is given.

scholarly journals SCHEDULING IN A SINGLE-SERVER QUEUE WITH STATE-DEPENDENT SERVICE RATES

2019 ◽  
Vol 34 (4) ◽  
pp. 507-521
Author(s):  
Urtzi Ayesta ◽  
Balakrishna Prabhu ◽  
Rhonda Righter
Keyword(s):  
Optimal Policy ◽  
Arrival Rate ◽  
Complete Characterization ◽  
Single Server ◽  
Holding Costs ◽  
Release Times ◽  
State Dependent ◽  
Service Rates ◽  
Transient System ◽  
The Cost

We consider single-server scheduling to minimize holding costs where the capacity, or rate of service, depends on the number of jobs in the system, and job sizes become known upon arrival. In general, this is a hard problem, and counter-intuitive behavior can occur. For example, even with linear holding costs the optimal policy may be something other than SRPT or LRPT, it may idle, and it may depend on the arrival rate. We first establish an equivalence between our problem of deciding which jobs to serve when completed jobs immediately leave, and a problem in which we have the option to hold on to completed jobs and can choose when to release them, and in which we always serve jobs according to SRPT. We thus reduce the problem to determining the release times of completed jobs. For the clearing, or transient system, where all jobs are present at time 0, we give a complete characterization of the optimal policy and show that it is fully determined by the cost-to-capacity ratio. With arrivals, the problem is much more complicated, and we can obtain only partial results.

EXISTENCE OF OPTIMAL STATIONARY POLICIES IN FINITE DYNAMIC PROGRAMS WITH NONNEGATIVE REWARDS

2001 ◽  
Vol 15 (4) ◽  
pp. 557-564 ◽  
Author(s):  
Rolando Cavazos-Cadena ◽  
Raúl Montes-de-Oca
Keyword(s):  
Control Policy ◽  
Stationary Policy ◽  
Reward Function ◽  
Total Reward ◽  
Dynamic Programs ◽  
Finite State ◽  
Markov Decision ◽  
Optimal Stationary Policy ◽  
Action Spaces ◽  
Discounted Criterion

This article concerns Markov decision chains with finite state and action spaces, and a control policy is graded via the expected total-reward criterion associated to a nonnegative reward function. Within this framework, a classical theorem guarantees the existence of an optimal stationary policy whenever the optimal value function is finite, a result that is obtained via a limit process using the discounted criterion. The objective of this article is to present an alternative approach, based entirely on the properties of the expected total-reward index, to establish such an existence result.

Sign in / Sign up

Export Citation Format

Share Document