The emergence of blockchains has fueled the development of resilient systems that can deal with
Byzantine failures
due to crashes, bugs, or even malicious behavior. Recently, we have also seen the exploration of
sharding
in these resilient systems, this to provide the scalability required by very large data-based applications. Unfortunately, current sharded resilient systems all use system-specific specialized approaches toward sharding that do not provide the flexibility of traditional sharded data management systems.
To improve on this situation, we fundamentally look at the design of sharded resilient systems. We do so by introducing BYSHARD, a unifying framework for the study of sharded resilient systems. Within this framework, we show how
two-phase commit
and
two-phase locking
---two techniques central to providing
atomicity
and
isolation
in traditional sharded databases---can be implemented efficiently in a Byzantine environment, this with a minimal usage of costly Byzantine resilient primitives. Based on these techniques, we propose
eighteen
multi-shard transaction processing protocols. Finally, we practically evaluate these protocols and show that each protocol supports high transaction throughput and provides scalability while each striking its own trade-off between
throughput, isolation level, latency
, and
abort rate.
As such, our work provides a strong foundation for the development of ACID-compliant general-purpose and flexible sharded resilient data management systems.
An Optimal Initial Partitioning of Large Data Model in Utility Management Systems