Evaluating asynchronous Schwarz solvers on GPUs

With the commencement of the exascale computing era, we realize that the majority of the leadership supercomputers are heterogeneous and massively parallel. Even a single node can contain multiple co-processors such as GPUs and multiple CPU cores. For example, ORNL’s Summit accumulates six NVIDIA Tesla V100 GPUs and 42 IBM Power9 cores on each node. Synchronizing across compute resources of multiple nodes can be prohibitively expensive. Hence, it is necessary to develop and study asynchronous algorithms that circumvent this issue of bulk-synchronous computing. In this study, we examine the asynchronous version of the abstract Restricted Additive Schwarz method as a solver. We do not explicitly synchronize, but allow the communication between the sub-domains to be completely asynchronous, thereby removing the bulk synchronous nature of the algorithm. We accomplish this by using the one-sided Remote Memory Access (RMA) functions of the MPI standard. We study the benefits of using such an asynchronous solver over its synchronous counterpart. We also study the communication patterns governed by the partitioning and the overlap between the sub-domains on the global solver. Finally, we show that this concept can render attractive performance benefits over the synchronous counterparts even for a well-balanced problem.

A Relaxation of Üresin and Dubois’ Asynchronous Fixed-Point Theory in Agda

AbstractÜresin and Dubois’ paper “Parallel Asynchronous Algorithms for Discrete Data” shows how a class of synchronous iterative algorithms may be transformed into asynchronous iterative algorithms. They then prove that the correctness of the resulting asynchronous algorithm can be guaranteed by reasoning about the synchronous algorithm alone. These results have been used to prove the correctness of various distributed algorithms, including in the fields of routing, numerical analysis and peer-to-peer protocols. In this paper we demonstrate several ways in which the assumptions that underlie this theory may be relaxed. Amongst others, we (i) expand the set of schedules for which the asynchronous iterative algorithm is known to converge and (ii) weaken the conditions that users must prove to hold to guarantee convergence. Furthermore, we demonstrate that two of the auxiliary results in the original paper are incorrect, and explicitly construct a counter-example. Finally, we also relax the alternative convergence conditions proposed by Gurney based on ultrametrics. Many of these relaxations and errors were uncovered after formalising the work in the proof assistant Agda. This paper describes the Agda code and the library that has resulted from this work. It is hoped that the library will be of use to others wishing to formally verify the correctness of asynchronous iterative algorithms.

Asynchronous Algorithms for Computing Equilibrium Prices in a Capital Asset Pricing Model

In this paper, we extend the work of [Tong, J, J Hu and J Hu (2017). Computing equilibrium prices for a capital asset pricing model with heterogeneous beliefs and margin-requirement constraints. European Journal of Operational Research, 256(1), 24–34] and develop various asynchronous algorithms to calculate the equilibrium asset prices in a heterogeneous capital asset pricing model. These asynchronous algorithms are based on different asynchronous updating schemes such as delayed updating, cyclic updating, fixed-length updating and random updating. In addition to potential benefits of improving computational efficiency, these asynchronous updating schemes also reflect several scenarios in financial markets in which investors may receive asset pricing information with various degrees of delays and their preferences on how and when to rebalance their portfolios may also be different. The proofs for the convergence of these algorithms are given. Numerical experiments are also provided to compare these algorithms and they show that these asynchronous algorithms work quite well.