A Study on Distributed Optimization over Large-Scale Networked Systems

Distributed optimization is a very important concept with applications in control theory and many related fields, as it is high fault-tolerant and extremely scalable compared with centralized optimization. Centralized solution methods are not suitable for many application domains that consist of large number of networked systems. In general, these large-scale networked systems cooperatively find an optimal solution to a common global objective during the optimization process. Thus, it gives us an opportunity to analyze distributed optimization techniques that is demanded in most distributed optimization settings. This paper presents an analysis that provides an overview of decomposition methods as well as currently existing distributed methods and techniques that are employed in large-scale networked systems. A detailed analysis on gradient like methods, subgradient methods, and methods of multipliers including the alternating direction method of multipliers is presented. These methods are analyzed empirically by using numerical examples. Moreover, an example highlighting the fact that the gradient method fails to solve distributed problems in some circumstances is discussed under numerical results. A numerical implementation is used to demonstrate that the alternating direction method of multipliers can solve this particular problem, by revealing its robustness compared with the gradient method. Finally, we conclude the paper with possible future research directions.

Parallelizing Subgradient Methods for the Lagrangian Dual in Stochastic Mixed-Integer Programming

The dual decomposition of stochastic mixed-integer programs can be solved by the projected subgradient algorithm. We show how to make this algorithm more amenable to parallelization in a master-worker model by describing two approaches, which can be combined in a natural way. The first approach partitions the scenarios into batches and makes separate use of subgradient information for each batch. The second approach drops the requirement that evaluation of function and subgradient information is synchronized across the scenarios. We provide convergence analysis of both methods. We also evaluate their performance on two families of problems from SIPLIB on a single server with 32 single-core worker processes, demonstrating that when the number of workers is high relative to the number of scenarios, these two approaches (and their synthesis) can significantly reduce running time.

A Novel Lagrangian Multiplier Update Algorithm for Short-Term Hydro-Thermal Coordination

The backbone of a conventional electrical power generation system relies on hydro-thermal coordination. Due to its intrinsic complex, large-scale and constrained nature, the feasibility of a direct approach is reduced. With this limitation in mind, decomposition methods, particularly Lagrangian relaxation, constitutes a consolidated choice to “simplify” the problem. Thus, translating a relaxed problem approach indirectly leads to solutions of the primal problem. In turn, the dual problem is solved iteratively, and Lagrange multipliers are updated between each iteration using subgradient methods. However, this class of methods presents a set of sensitive aspects that often require time-consuming tuning tasks or to rely on the dispatchers’ own expertise and experience. Hence, to tackle these shortcomings, a novel Lagrangian multiplier update adaptative algorithm is proposed, with the aim of automatically adjust the step-size used to update Lagrange multipliers, therefore avoiding the need to pre-select a set of parameters. A results comparison is made against two traditionally employed step-size update heuristics, using a real hydrothermal scenario derived from the Portuguese power system. The proposed adaptive algorithm managed to obtain improved performances in terms of the dual problem, thereby reducing the duality gap with the optimal primal problem.

Short-term Hydro-thermal Coordination By Lagrangian Relaxation: A New Algorithm for the Solution of the Dual Problem

For decades, researchers have been studying the unit commitment problem in electrical power generation. To solve this complex, large scale and constrained optimization (primal) problem in a direct manner is not a feasible approach, which is where Lagrangian relaxation comes in as the answer. The dual Lagrangian problem translates a relaxed problem approach, that indirectly leads to solutions of the original (primal) problem. In the coordination problem, a decomposition takes place where the global solution is achieved by coordinating between the respective subproblems solutions. This dual problem is solved iteratively, and Lagrange multipliers are updated between each iteration using subgradient methods. To tackle, time-consuming tuning tasks or user related experience, a new adaptative algorithm, is proposed to better adjust the step-size used to update Lagrange multipliers, i.e., avoid the need to pre-select a set of parameters. A results comparison against a traditionally employed step-size update mechanism, showed that the adaptive algorithm manages to obtain improved performances in terms of the targeted primal problem. Keywords: Hydro-Thermal coordination, Lagrangian relaxation, Lagrangian dual problem, Lagrange multipliers, Subgradient methods