Distributionally Robust Markov Decision Processes and Their Connection to Risk Measures

We consider robust Markov decision processes with Borel state and action spaces, unbounded cost, and finite time horizon. Our formulation leads to a Stackelberg game against nature. Under integrability, continuity, and compactness assumptions, we derive a robust cost iteration for a fixed policy of the decision maker and a value iteration for the robust optimization problem. Moreover, we show the existence of deterministic optimal policies for both players. This is in contrast to classical zero-sum games. In case the state space is the real line, we show under some convexity assumptions that the interchange of supremum and infimum is possible with the help of Sion’s minimax theorem. Further, we consider the problem with special ambiguity sets. In particular, we are able to derive some cases where the robust optimization problem coincides with the minimization of a coherent risk measure. In the final section, we discuss two applications: a robust linear-quadratic problem and a robust problem for managing regenerative energy.

On the Optimality of Affine Policies for Budgeted Uncertainty Sets

In this paper, we study the performance of affine policies for a two-stage, adjustable, robust optimization problem with a fixed recourse and an uncertain right-hand side belonging to a budgeted uncertainty set. This is an important class of uncertainty sets, widely used in practice, in which we can specify a budget on the adversarial deviations of the uncertain parameters from the nominal values to adjust the level of conservatism. The two-stage adjustable robust optimization problem is hard to approximate within a factor better than [Formula: see text] even for budget of uncertainty sets in which [Formula: see text] is the number of decision variables. Affine policies, in which the second-stage decisions are constrained to be an affine function of the uncertain parameters provide a tractable approximation for the problem and have been observed to exhibit good empirical performance. We show that affine policies give an [Formula: see text]-approximation for the two-stage, adjustable, robust problem with fixed nonnegative recourse for budgeted uncertainty sets. This matches the hardness of approximation, and therefore, surprisingly, affine policies provide an optimal approximation for the problem (up to a constant factor). We also show strong theoretical performance bounds for affine policy for a significantly more general class of intersection of budgeted sets, including disjoint constrained budgeted sets, permutation invariant sets, and general intersection of budgeted sets. Our analysis relies on showing the existence of a near-optimal, feasible affine policy that satisfies certain nice structural properties. Based on these structural properties, we also present an alternate algorithm to compute a near-optimal affine solution that is significantly faster than computing the optimal affine policy by solving a large linear program.

Distributionally robust multi-period portfolio selection subject to bankruptcy constraints

<p style='text-indent:20px;'>An optimization problem with moments information which suffers from distributional uncertainty can be handled through distributionally robust optimization. In this paper, we will consider distributionally robust multi-period portfolio selection since only moment information of portfolios can be gathered in practice. We will consider two different scenarios. One is that moments information can be obtained exactly and the other one is that the moments information is also uncertain. For the two scenarios, we will show how to transform the corresponding distributionally robust optimization problem into a second order cone problem (SOCP) which can be easily solved by existing methods. Some numerical experiments are presented to demonstrate the effectiveness of our proposed method.</p>

Unsupervised Learning and Robust Optimization in the Portfolio Problem

Robust optimization in various problems of science and technology arises from the uncertainty of the parameters that determine the decision-making model. The parameters can be judged on the basis of a large number of observed examples. This challenge is to customize a solution based on a large number of examples that works well for examples that were not involved in customizing the solution. In this sense, the robust optimization problem belongs to machine learning problems. The paper uses a dichotomous clustering algorithm to determine the range of parameters for the optimal portfolio problem.

On optimality conditions for robust weak sharp solution in uncertain optimizations

In this paper, we investigate the robust optimization problem involving nonsmooth and nonconvex real-valued functions. We firstly establish a necessary condition for the local robust weak sharp solution of considered problem under a constraint qualification. These optimality conditions are presented in terms of multipliers and Mordukhovich subdifferentials of the related functions. Then, by employing the robust version of the (KKT) condition, and some appropriate generalized convexity conditions, we also obtain some sufficient conditions for the global robust weak sharp solutions of the problem. In addition, some examples are presented for illustrating or supporting the results.

Robust Optimization for Electricity Generation

We consider a robust optimization problem in an electric power system under uncertain demand and availability of renewable energy resources. Solving the deterministic alternating current (AC) optimal power flow (ACOPF) problem has been considered challenging since the 1960s due to its nonconvexity. Linear approximation of the AC power flow system sees pervasive use, but does not guarantee a physically feasible system configuration. In recent years, various convex relaxation schemes for the ACOPF problem have been investigated, and under some assumptions, a physically feasible solution can be recovered. Based on these convex relaxations, we construct a robust convex optimization problem with recourse to solve for optimal controllable injections (fossil fuel, nuclear, etc.) in electric power systems under uncertainty (renewable energy generation, demand fluctuation, etc.). We propose a cutting-plane method to solve this robust optimization problem, and we establish convergence and other desirable properties. Experimental results indicate that our robust convex relaxation of the ACOPF problem can provide a tight lower bound.

Evaluation of Wind Energy Accommodation Based on Two-Stage Robust Optimization

Large-scale renewable energy integration brings unprecedented challenges to electric power system planning and operation. The paper aims at economic dispatch and the safe operation of high penetration renewable energy power systems. According to the principle of power system dispatchability, the assessment of wind energy accommodation is formulated into a two-stage robust optimization problem with a min-max-min structure. Based on the benders algorithm, the intractable robust optimization problem is transformed into the form of sub-problem and master problem. Strong duality theory and big-M method are used to recast the sub problem into a mixed integer linear programming. The envelope of wind energy accommodation can be obtained by using commercial software to solve the master problem and sub problem alternately. For the realization of arbitrary wind power within the envelope, the amount of wind energy leakage and load shedding in power system operation are acceptable. An example of modified IEEE 39-bus test systems is used to verify the effectiveness and practicability of the evaluation method.