Learning When to Commit: Schedule Distillation for Fast Diffusion Language Models
dSpark distills token reveal schedules for diffusion language models, aiming to cut decoding rounds while preserving task performance. Its reported speedups may interest builders exploring parallel generation for code and reasoning.
dSpark learns which token reveals make other positions easier to predict, then distills that schedule into diffusion language models. A **KL-divergence regularizer** is used to help preserve generation quality. On GSM8K, the paper reports **10.12×** speedup for LLaDA-8B-Instruct and **5.12×** for Dream-7B-Instruct over various baselines.
Builders evaluating diffusion models for code or reasoning generation can track whether schedule learning reduces latency without hurting task results.
dSpark learns which token reveals make other positions easier to predict, then distills that schedule into diffusion language models. A **KL-divergence regularizer** is used to help preserve generation quality. On GSM8K, the paper reports **10.12×** speedup for LLaDA-8B-Instruct and **5.12×** for Dream-7B-Instruct over various baselines. Builders evaluating diffusion models for code or reasoning generation can track whether schedule learning reduces latency without hurting task results. The reported speedups are for GSM8K and use various baselines; the material does not give a direct comparison of latency or quality on code-generation tasks.
dSpark targets diffusion-model latency by learning token reveal order and distilling it, with a KL regularizer to help preserve generation quality. The reported GSM8K speedups motivate a code-task check, but do not establish that the gains transfer to coding or reasoning workloads beyond that benchmark.