A benchmark for evaluating code-level diagnosis of distributed-system failures.
DistDebug-Bench evaluates whether language-model agents can diagnose real distributed-system failures from issue reports, repository snapshots, and project context. The benchmark focuses on finding the source-code blocks that explain a reported failure before the fixing patch is applied.
The benchmark contains 120 bug reports from 11 distributed systems and 5 programming languages, with gold root-cause labels for code-level evaluation.
Benchmark repository: github.com/OrderLab/distdebug-bench
Issue evidence, repository snapshots, gold root-cause blocks, and evaluation scripts.
Original issue descriptions and metadata from real distributed-system bug reports.
Gold source-code blocks that capture the pre-fix root cause for each failure.
Scoring logic for comparing predicted diagnosis blocks against human labels.