scholarly journals LTL-Constrained Steady-State Policy Synthesis

2021 ◽  
Author(s):  
Jan Křetínský
Keyword(s):  
Steady State ◽  
State Policy ◽  
Infinite Horizon ◽  
Long Run ◽  
Markov Decision ◽  
Quantitative Properties ◽  
The One ◽  
The Given ◽  
Type Specification ◽  
Ltl Model Checking

Decision-making policies for agents are often synthesized with the constraint that a formal specification of behaviour is satisfied. Here we focus on infinite-horizon properties. On the one hand, Linear Temporal Logic (LTL) is a popular example of a formalism for qualitative specifications. On the other hand, Steady-State Policy Synthesis (SSPS) has recently received considerable attention as it provides a more quantitative and more behavioural perspective on specifications, in terms of the frequency with which states are visited. Finally, rewards provide a classic framework for quantitative properties. In this paper, we study Markov decision processes (MDP) with the specification combining all these three types. The derived policy maximizes the reward among all policies ensuring the LTL specification with the given probability and adhering to the steady-state constraints. To this end, we provide a unified solution reducing the multi-type specification to a multi-dimensional long-run average reward. This is enabled by Limit-Deterministic Büchi Automata (LDBA), recently studied in the context of LTL model checking on MDP, and allows for an elegant solution through a simple linear programme. The algorithm also extends to the general omega-regular properties and runs in time polynomial in the sizes of the MDP as well as the LDBA.

Optimal service-rate control of M/G/1 queueing systems using phase methods

1983 ◽  
Vol 15 (3) ◽  
pp. 616-637 ◽  
Author(s):  
Kyung Y. Jo ◽  
Shaler Stidham
Keyword(s):  
Partial Information ◽  
Queueing Systems ◽  
Infinite Horizon ◽  
Service Rate ◽  
Erlang Distribution ◽  
Work Requirement ◽  
Average Return ◽  
Long Run ◽  
Markov Decision ◽  
Method Of Phases

A new approach to the optimal control of the service rate in M/G/1 queues is introduced using the method of phases. Each customer's work requirement is approximated by a random number of exponential phases with (possibly) different parameters (a generalized hyper-Erlang distribution). Using a semi-Markov decision-process formulation, we establish monotonicity properties of optimal policies for the finite-horizon problem, by induction on the horizon length. The analysis is then extended to the discounted infinite-horizon and the long-run average-return problems. In contrast to the models in previous papers, our model is appropriate for situations where the system controller has partial information about the work requirement of a customer, specifically the number of phases (tasks) to be performed. Because it requires a multidimensional state description, the analysis of the phase-type control model may be viewed as a first step toward the solution of control models for networks of queues.

scholarly journals Steady-State Policy Synthesis for Verifiable Control

2019 ◽  
Author(s):  
Alvaro Velasquez
Keyword(s):  
Decision Making ◽  
Linear Programming ◽  
Steady State ◽  
Markov Decision Processes ◽  
State Policy ◽  
Decision Processes ◽  
State Control ◽  
Steady State Probability ◽  
Markov Decision ◽  
Steady State Control

In this paper, we introduce the Steady-State Policy Synthesis (SSPS) problem which consists of finding a stochastic decision-making policy that maximizes expected rewards while satisfying a set of asymptotic behavioral specifications. These specifications are determined by the steady-state probability distribution resulting from the Markov chain induced by a given policy. Since such distributions necessitate recurrence, we propose a solution which finds policies that induce recurrent Markov chains within possibly non-recurrent Markov Decision Processes (MDPs). The SSPS problem functions as a generalization of steady-state control, which has been shown to be in PSPACE. We improve upon this result by showing that SSPS is in P via linear programming. Our results are validated using CPLEX simulations on MDPs with over 10000 states. We also prove that the deterministic variant of SSPS is NP-hard.

scholarly journals An optimal broker design for inter-cloud systems

2021 ◽  
Author(s):  
Shraddha R Peesary
Keyword(s):  
Service Providers ◽  
Cloud Service ◽  
End Users ◽  
Service Request ◽  
Cloud Systems ◽  
Cloud Broker ◽  
Cost Criterion ◽  
Long Run ◽  
Markov Decision ◽  
The One

In the next generation of Cloud computing systems, it is expected that multiple Cloud Service Providers (CSPs) will cooperate together to advertise their services and prices to their end users, which may choose the one that best meets their budgetary and technical needs. Despite this benefit of having multiple CSPs to select from, several issues may arise. For instance, how does an IT entrepreneur select a CSP to offload his/her service request? How does the underlying Inter-Cloud system handle this service request? To address these questions, this thesis proposes a novel Optimal Cloud Broker design for Inter-Cloud Systems in the form of a Semi-Markov Decision Process (SMDP) based model. Under the long-run expected average cost criterion, the optimal policy is derived, which aim at maximizing the overall virtual machine utilization while giving the end users the best possible prices. The effectiveness of the proposed Broker design is validated by numerical results.

scholarly journals Steady-State Policy Synthesis in Multichain Markov Decision Processes

2020 ◽  
Author(s):  
George Atia ◽  
Andre Beckus ◽  
Ismail Alkhouri ◽  
Alvaro Velasquez
Keyword(s):  
Steady State ◽  
Markov Decision Processes ◽  
State Policy ◽  
Autonomous Agents ◽  
Decision Processes ◽  
Intelligence Community ◽  
State Control ◽  
Markov Decision ◽  
Feasible Solutions ◽  
Steady State Control

The formal synthesis of automated or autonomous agents has elicited strong interest from the artificial intelligence community in recent years. This problem space broadly entails the derivation of decision-making policies for agents acting in an environment such that a formal specification of behavior is satisfied. Popular formalisms for such specifications include the quintessential Linear Temporal Logic (LTL) and Computation Tree Logic (CTL) which reason over infinite sequences and trees, respectively, of states. However, the related and relevant problem of reasoning over the frequency with which states are visited infinitely and enforcing behavioral specifications on the same has received little attention. That problem, known as Steady-State Policy Synthesis (SSPS) or steady-state control, is the focus of this paper. Prior related work has been mostly confined to unichain Markov Decision Processes (MDPs), while a tractable solution to the general multichain setting heretofore remains elusive. In this paper, we provide a solution to the latter within the context of multichain MDPs over a class of policies that account for all possible transitions in the given MDP. The solution policy is derived from a novel linear program (LP) that encodes constraints on the limiting distributions of the Markov chain induced by said policy. We establish a one-to-one correspondence between the feasible solutions of the LP and the stationary distributions of the induced Markov chains. The derived policy is shown to maximize the reward among the constrained class of stationary policies and to satisfy the specification constraints even when it does not exercise all possible transitions.

scholarly journals An optimal broker design for inter-cloud systems

2021 ◽  
Author(s):  
Shraddha R Peesary
Keyword(s):  
Service Providers ◽  
Cloud Service ◽  
End Users ◽  
Service Request ◽  
Cloud Systems ◽  
Cloud Broker ◽  
Cost Criterion ◽  
Long Run ◽  
Markov Decision ◽  
The One

In the next generation of Cloud computing systems, it is expected that multiple Cloud Service Providers (CSPs) will cooperate together to advertise their services and prices to their end users, which may choose the one that best meets their budgetary and technical needs. Despite this benefit of having multiple CSPs to select from, several issues may arise. For instance, how does an IT entrepreneur select a CSP to offload his/her service request? How does the underlying Inter-Cloud system handle this service request? To address these questions, this thesis proposes a novel Optimal Cloud Broker design for Inter-Cloud Systems in the form of a Semi-Markov Decision Process (SMDP) based model. Under the long-run expected average cost criterion, the optimal policy is derived, which aim at maximizing the overall virtual machine utilization while giving the end users the best possible prices. The effectiveness of the proposed Broker design is validated by numerical results.

Optimal service-rate control of M/G/1 queueing systems using phase methods

1983 ◽  
Vol 15 (03) ◽  
pp. 616-637 ◽  
Author(s):  
Kyung Y. Jo ◽  
Shaler Stidham
Keyword(s):  
Partial Information ◽  
Queueing Systems ◽  
Infinite Horizon ◽  
Service Rate ◽  
Erlang Distribution ◽  
Work Requirement ◽  
Average Return ◽  
Long Run ◽  
Markov Decision ◽  
Method Of Phases

A new approach to the optimal control of the service rate in M/G/1 queues is introduced using the method of phases. Each customer's work requirement is approximated by a random number of exponential phases with (possibly) different parameters (a generalized hyper-Erlang distribution). Using a semi-Markov decision-process formulation, we establish monotonicity properties of optimal policies for the finite-horizon problem, by induction on the horizon length. The analysis is then extended to the discounted infinite-horizon and the long-run average-return problems. In contrast to the models in previous papers, our model is appropriate for situations where the system controller has partial information about the work requirement of a customer, specifically the number of phases (tasks) to be performed. Because it requires a multidimensional state description, the analysis of the phase-type control model may be viewed as a first step toward the solution of control models for networks of queues.

Migration and US agricultural competitiveness

2013 ◽  
Vol 10 (2) ◽  
pp. 159-179 ◽  
Author(s):  
Philip L. Martin
Keyword(s):  
Farm Workers ◽  
Easy Access ◽  
Foreign Born ◽  
Guest Workers ◽  
Long Run ◽  
Short Run ◽  
Lower Wage ◽  
The Government ◽  
The One

Agriculture has one of the highest shares of foreign-born and unauthorized workers among US industries; over three-fourths of hired farm workers were born abroad, usually in Mexico, and over half of all farm workers are unauthorized. Farm employers are among the few to openly acknowledge their dependence on migrant and unauthorized workers, and they oppose efforts to reduce unauthorized migration unless the government legalizes currently illegal farm workers or provides easy access to legal guest workers. The effects of migrants on agricultural competitiveness are mixed. On the one hand, wages held down by migrants keep labour-intensive commodities competitive in the short run, but the fact that most labour-intensive commodities are shipped long distances means that long-run US competitiveness may be eroded as US farmers have fewer incentives to develop labour-saving and productivity-improving methods of farming and production in lower-wage countries expands.

THE EU SUGAR MARKET PROFILE AND ITS MAIN DRIVERS

2015 ◽  
Author(s):  
Lubos SMUTKA ◽  
Irena BENEŠOVÁ ◽  
Patrik ROVNÝ ◽  
Renata MATYSIK-PEJAS
Keyword(s):  
Market Failure ◽  
Current System ◽  
Quota System ◽  
Considerable Debate ◽  
Sugar Market ◽  
The Common ◽  
The One ◽  
Production Capacities ◽  
The Given ◽  
The Eu

Sugar is one of the most important elements in human nutrition. The Common Market Organisation for sugar has been a subject of considerable debate since its establishment in 1968. The European agricultural market has been criticized for its heavy regulations and subsidization. The sugar market is one of the most regulated ones; however, this will change radically in 2017 when the current system of production quotas will end. The current EU sugar market changed is structure during the last several decades. The significant number of companies left the market and EU internal sugar market became more concentrated. The aim of this paper is presentation characteristics of sugar market with respect to the supposed market failure – reduction in competition. The analysis also identifies the main drivers and determinants of the EU especially quota sugar market. In relation to paper’s aim the following results are important. The present conditions of the European sugar market have led to market failure when nearly 75 % (10 million tonnes) of the quota is controlled by five multinational companies only. These multinational alliances (especially German and French one) are also taking control over the production capacities of their subsidiaries. In most countries, this causes serious problems as the given quota is controlled by one or two producers only. This is a significant indicator of market imperfection. The quota system cannot overcome the problem of production quotas on the one hand and the demand on the other; furthermore, it also leads to economic inefficiency. The current EU sugar market is under the control of only Sudzucker, Nordzucker, Pfeifer and Langen, Tereos and ABF.

Quasi-Periodic Control Technique for Minimizing Fuel Consumption During Record Vehicle Competition

2013 ◽  
Vol 60 (2) ◽  
pp. 185-197 ◽  
Author(s):  
Paweł Sulikowski ◽  
Ryszard Maronski
Keyword(s):  
Fuel Consumption ◽  
Optimal Control Theory ◽  
Slope Angle ◽  
Control Technique ◽  
Decision Variables ◽  
Aeronautical Engineering ◽  
The One ◽  
Optimal Driving ◽  
The Given ◽  
Engine Power

The problem of the optimal driving technique during the fuel economy competition is reconsidered. The vehicle is regarded as a particle moving on a trace with a variable slope angle. The fuel consumption is minimized as the vehicle covers the given distance in a given time. It is assumed that the run consists of two recurrent phases: acceleration with a full available engine power and coasting down with the engine turned off. The most fuel-efficient technique for shifting gears during acceleration is found. The decision variables are: the vehicle velocities at which the gears should be shifted, on the one hand, and the vehicle velocities when the engine should be turned on and off, on the other hand. For the data of students’ vehicle representing the Faculty of Power and Aeronautical Engineering it has been found that such driving strategy is more effective in comparison with a constant speed strategy with the engine partly throttled, as well as a strategy resulting from optimal control theory when the engine is still active.

First grammar books in the Habsburg Monarchy: individual initiative and regulatory interference by the state (1760s–1770s)

2020 ◽  
Vol 2019 (2 (11)) ◽  
pp. 137-157
Author(s):  
Olga V. Khavanova ◽  
Keyword(s):  
Eighteenth Century ◽  
State Policy ◽  
Large Scale ◽  
Language Use ◽  
Linguistic Diversity ◽  
Mother Tongue ◽  
German Language ◽  
Habsburg Monarchy ◽  
Private Initiative ◽  
The One

The second half of the eighteenth century in the lands under the sceptre of the House of Austria was a period of development of a language policy addressing the ethno-linguistic diversity of the monarchy’s subjects. On the one hand, the sphere of use of the German language was becoming wider, embracing more and more segments of administration, education, and culture. On the other hand, the authorities were perfectly aware of the fact that communication in the languages and vernaculars of the nationalities living in the Austrian Monarchy was one of the principal instruments of spreading decrees and announcements from the central and local authorities to the less-educated strata of the population. Consequently, a large-scale reform of primary education was launched, aimed at making the whole population literate, regardless of social status, nationality (mother tongue), or confession. In parallel with the centrally coordinated state policy of education and language-use, subjects-both language experts and amateur polyglots-joined the process of writing grammar books, which were intended to ease communication between the different nationalities of the Habsburg lands. This article considers some examples of such editions with primary attention given to the correlation between private initiative and governmental policies, mechanisms of verifying the textbooks to be published, their content, and their potential readers. This paper demonstrates that for grammar-book authors, it was very important to be integrated into the patronage networks at the court and in administrative bodies and stresses that the Vienna court controlled the process of selection and financing of grammar books to be published depending on their quality and ability to satisfy the aims and goals of state policy.

Sign in / Sign up

Export Citation Format

Share Document