Benchmarking Apache Kafka under network faults

Abstract

Network faults are often transient and hence hard to detect and difficult to resolve. Our study conducts an analysis of Kafka's network fault tolerance capabilities, one of the widely used distributed stream processing system (DSPS). Across different Kafka configurations, we observed that Kafka is fault-tolerant towards network faults to some degree, and we report observations of its shortcomings. We also define a network fault-tolerance benchmark on which other DSPSs can be evaluated.