Skip to content

Diablo: A Benchmark Suite for Blockchains

Eurosys 2023

Authored by:
Vincent Gramoli, University of Sydney, CTO Redbelly Network
Rachid Guerraoui, Swiss Federal Institute of Technology
Andrei Lebedev, Technical University of Munich
Chris Natoli, University of Sydney
Gauthier Voron, Swiss Federal Institute of Technology


With the recent advent of blockchains, we have witnessed a plethora of blockchain proposals. These proposals range from using work to using time, storage or stake in order to select blocks to be appended to the chain. As a drawback it makes it difficult for the application developer to choose the right blockchain to support their applications. In particular, the scalability and performance one can obtain from a specific blockchain is typically unknown. The claimed results are often obtained in isolation by the developers of the blockchain themselves. The experimental conditions corresponding to these results are generally missing and the lack of details make these results irreproducible.

In this paper, we propose the most extensive evaluation of blockchain to date. First, we show how the experimental settings impact the performance of 6 state-of-the-art blockchains and argue for more detailed experiments. Second, and to cope with this limitation, we propose a unifying framework to evaluate blockchains on the same ground. The framework includes a suite of 5 realistic Decentralised Applications (DApps), helps deploy the blockchain nodes at different scales and evaluate their performance. Finally, we show that selecting a particular virtual machine or weakening guarantees can help handle computationally demanding workloads but that none of the tested blockchains can yet support the load of these realistic DApps.

Redbelly Network Pty Ltd

304/74 Pitt St. Sydney 
NSW 2000, Australia

2nd floor, Plot 14, Aeren Building, 
IT park Chandigarh, sector 13, Chandigarh, India