Fault Tolerance Techniques for Distributed, Parallel Applications

2016 ◽  
pp. 221-252
Author(s):  
Camille Coti
Keyword(s):  
Fault Tolerance ◽  
High Performance Computing ◽  
High Performance ◽  
Fault Tolerant ◽  
Distributed Applications ◽  
Parallel Applications ◽  
Rollback Recovery ◽  
Tolerance Mechanisms ◽  
Performance Computing

This chapter gives an overview of techniques used to tolerate failures in high-performance distributed applications. We describe basic replication techniques, automatic rollback recovery and application-based fault tolerance. We present the challenges raised specifically by distributed, high performance computing and the performance overhead the fault tolerance mechanisms are likely to cost. Last, we give an example of a fault-tolerant algorithm that exploits specific properties of a recent algorithm.

scholarly journals Transparent fault tolerance for scalable functional computation

2016 ◽  
Vol 26 ◽  
Author(s):  
ROBERT STEWART ◽  
PATRICK MAIER ◽  
PHIL TRINDER
Keyword(s):  
Fault Tolerance ◽  
High Performance Computing ◽  
High Performance ◽  
Large Scale ◽  
Programming Model ◽  
Fault Tolerant ◽  
Fault Recovery ◽  
Actor Model ◽  
Work Stealing ◽  
Performance Computing

AbstractReliability is set to become a major concern on emergent large-scale architectures. While there are many parallel languages, and indeed many parallel functional languages, very few address reliability. The notable exception is the widely emulated Erlang distributed actor model that provides explicit supervision and recovery of actors with isolated state. We investigate scalable transparent fault tolerant functional computation with automatic supervision and recovery of tasks. We do so by developing HdpH-RS, a variant of the Haskell distributed parallel Haskell (HdpH) DSL with Reliable Scheduling. Extending the distributed work stealing protocol of HdpH for task supervision and recovery is challenging. To eliminate elusive concurrency bugs, we validate the HdpH-RS work stealing protocol using the SPIN model checker. HdpH-RS differs from the actor model in that its principal entities are tasks, i.e. independent stateless computations, rather than isolated stateful actors. Thanks to statelessness, fault recovery can be performed automatically and entirely hidden in the HdpH-RS runtime system. Statelessness is also key for proving a crucial property of the semantics of HdpH-RS: fault recovery does not change the result of the program, akin to deterministic parallelism. HdpH-RS provides a simple distributed fork/join-style programming model, with minimal exposure of fault tolerance at the language level, and a library of higher level abstractions such as algorithmic skeletons. In fact, the HdpH-RS DSL is exactly the same as the HdpH DSL, hence users can opt in or out of fault tolerant execution without any refactoring. Computations in HdpH-RS are always as reliable as the root node, no matter how many nodes and cores are actually used. We benchmark HdpH-RS on conventional clusters and an High Performance Computing platform: all benchmarks survive Chaos Monkey random fault injection; the system scales well e.g. up to 1,400 cores on the High Performance Computing; reliability and recovery overheads are consistently low even at scale.

HIERARCHICAL REPLICATION TECHNIQUES TO ENSURE CHECKPOINT STORAGE RELIABILITY IN GRID ENVIRONMENT

2009 ◽  
Vol 10 (04) ◽  
pp. 345-364
Author(s):  
FATIHA BOUABACHE ◽  
THOMAS HERAULT ◽  
GILLES FEDAK ◽  
FRANCK CAPPELLO
Keyword(s):  
Fault Tolerance ◽  
High Performance Computing ◽  
High Performance ◽  
Rollback Recovery ◽  
Grid Environment ◽  
Replication Process ◽  
Mean Time Between Failures ◽  
Mpi Applications ◽  
Recovery Protocols ◽  
Performance Computing

An efficient and reliable fault tolerance protocol plays a key role in High Performance Computing. Rollback recovery is the most common fault tolerance technique used in High Performance Computing and especially in MPI applications. This technique relies on the reliability of the checkpoint storage. Most of the rollback recovery protocols assume that the checkpoint servers machines are reliable. However, in a grid environment any unit can fail at any moment, including components used to connect different administrative domains. Such failures lead to the loss of a whole set of machines, including the more reliable machines used to store the checkpoints in this administrative domain. Thus it is not safe to rely on the high Mean Time Between Failures of specific machines to store the checkpoint images. This paper introduces a new coordinated checkpoint protocol, which tolerates checkpoint server failures and clusters failures, and ensures a checkpoint storage reliability in a grid environment. To provide this reliability the protocol is based on a replication process. We propose new hierarchical replication strategies that exploit the locality of checkpoint images in order to minimize inter-cluster communication. We evaluate the effectiveness of our two hierarchical replication strategies through simulations against several criteria such as topology and scalability.

scholarly journals Application-based fault tolerance techniques for sparse matrix solvers

2017 ◽  
Vol 32 (5) ◽  
pp. 627-640
Author(s):  
Simon McIntosh–Smith ◽  
Rob Hunt ◽  
James Price ◽  
Alex Warwick Vesztrocy
Keyword(s):  
Fault Tolerance ◽  
High Performance Computing ◽  
High Performance ◽  
Sparse Matrix ◽  
Sparse Matrices ◽  
Error Correcting Codes ◽  
Computing Systems ◽  
Hardware Costs ◽  
Extreme Scale ◽  
Performance Computing

High-performance computing systems continue to increase in size in the quest for ever higher performance. The resulting increased electronic component count, coupled with the decrease in feature sizes of the silicon manufacturing processes used to build these components, may result in future exascale systems being more susceptible to soft errors caused by cosmic radiation than in current high-performance computing systems. Through the use of techniques such as hardware-based error-correcting codes and checkpoint-restart, many of these faults can be mitigated at the cost of increased hardware overhead, run-time, and energy consumption that can be as much as 10–20%. Some predictions expect these overheads to continue to grow over time. For extreme scale systems, these overheads will represent megawatts of power consumption and millions of dollars of additional hardware costs, which could potentially be avoided with more sophisticated fault-tolerance techniques. In this paper we present new software-based fault tolerance techniques that can be applied to one of the most important classes of software in high-performance computing: iterative sparse matrix solvers. Our new techniques enables us to exploit knowledge of the structure of sparse matrices in such a way as to improve the performance, energy efficiency, and fault tolerance of the overall solution.

The Sicilian Grid Infrastructure for High Performance Computing

2010 ◽  
Vol 1 (1) ◽  
pp. 40-54 ◽  
Author(s):  
Carmelo Marcello Iacono-Manno ◽  
Marco Fargetta ◽  
Roberto Barbera ◽  
Alberto Falzone ◽  
Giuseppe Andronico ◽  
...  
Keyword(s):  
High Performance Computing ◽  
High Performance ◽  
Parallel Applications ◽  
Grid Infrastructure ◽  
Scheduling Policy ◽  
Computing Paradigm ◽  
Regional Area ◽  
Computer Fluid Dynamics ◽  
Grid Infrastructures ◽  
Performance Computing

The conjugation of High Performance Computing (HPC) and Grid paradigm with applications based on commercial software is one among the major challenges of today e-Infrastructures. Several research communities from either industry or academia need to run high parallel applications based on licensed software over hundreds of CPU cores; a satisfactory fulfillment of such requests is one of the keys for the penetration of this computing paradigm into the industry world and sustainability of Grid infrastructures. This problem has been tackled in the context of the PI2S2 project that created a regional e-Infrastructure in Sicily, the first in Italy over a regional area. Present article will describe the features added in order to integrate an HPC facility into the PI2S2 Grid infrastructure, the adoption of the InifiniBand low-latency net connection, the gLite middleware extended to support MPI/MPI2 jobs, the newly developed license server and the specific scheduling policy adopted. Moreover, it will show the results of some relevant use cases belonging to Computer Fluid-Dynamics (Fluent, OpenFOAM), Chemistry (GAMESS), Astro-Physics (Flash) and Bio-Informatics (ClustalW)).

Sign in / Sign up

Export Citation Format

Share Document