Learning When Diffusion Language Models Should Regenerate
DiVR uses a verifier to choose when a diffusion language model should regenerate, raising average accuracy on five reasoning benchmarks while costing more than greedy decoding.
DiVR predicts whether a partially generated sequence will reach a correct answer, then chooses when to regenerate a block and which alternative to keep. With **LLaDA-8B-Base**, average accuracy across **five reasoning benchmarks** rises from **35.3% to 43.4%**, at **1.9–3.5×** greedy decoding cost.
For reasoning workloads using diffusion language models, token confidence alone may miss errors, and remasking one token can reproduce the same mistake. A verifier trained on the model’s own trajectories offers a way to decide when to revisit a larger block.
DiVR predicts whether a partially generated sequence will reach a correct answer, then chooses when to regenerate a block and which alternative to keep. With **LLaDA-8B-Base**, average accuracy across **five reasoning benchmarks** rises from **35.3% to 43.4%**, at **1.9–3.5×** greedy decoding cost. For reasoning workloads using diffusion language models, token confidence alone may miss errors, and remasking one token can reproduce the same mistake. A verifier trained on the model’s own trajectories offers a way to decide when to revisit a larger block. The reported gains are benchmark results and come with extra inference cost. The paper says the approach also works with Dream and complements post-training, but the material does not quantify those settings separately.
DiVR adds a learned decision about when to regenerate diffusion-generated blocks, with reported accuracy gains on five reasoning benchmarks and substantially higher decoding cost. It offers a quality-for-compute option, while the evidence remains specific to the tested model and benchmarks.