Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs
Flash-dLLM combines an I/O-aware KV-cache kernel with self-drafting parallel decoding, reporting large inference speedups without retraining or an auxiliary model.
Flash-dLLM is a training-free inference framework that couples a fused, I/O-aware KV-cache kernel with draft-and-verify decoding. The same diffusion LLM acts as drafter and verifier, reaching **5.1× on GSM8K** and **11.0× on HumanEval** over Elastic-Cache.
Builders evaluating diffusion models for coding or reasoning should profile GPU memory movement, not just compute, and test caching together with parallel verification. The design also removes the operational cost of serving a separate draft model.
Flash-dLLM is a training-free inference framework that couples a fused, I/O-aware KV-cache kernel with draft-and-verify decoding. The same diffusion LLM acts as drafter and verifier, reaching **5.1× on GSM8K** and **11.0× on HumanEval** over Elastic-Cache. Builders evaluating diffusion models for coding or reasoning should profile GPU memory movement, not just compute, and test caching together with parallel verification. The design also removes the operational cost of serving a separate draft model. The supplied results cover mathematical reasoning and code generation, but provide no absolute latency, hardware, batch-size, or quality figures. The gains therefore need validation against the intended serving workload.
This extends prior cache-aware serving guidance from routing and capacity planning into the diffusion decoder itself: I/O-aware caching and same-model draft verification can be optimized together without operating a second drafter. It strengthens the case that memory movement is a primary inference constraint, while the missing hardware, latency, batch, and quality details keep the reported speedups workload-specific.