arXivPaperNeeds Review
Audio-Native Speech Recognition with a Frozen Discrete-Diffusion Language Model
A frozen diffusion language model can transcribe speech by refining the full transcript in parallel. The prototype trains a small audio interface and reaches 6.6% WER in roughly eight steps.
arXiv · Jul 14, 2026
Source Summary
The system connects a frozen Whisper encoder to the **26B DiffusionGemma** backbone through a projector and low-rank adapters. It trains about **42M parameters**, or 0.16% of the backbone, and reaches **6.6% word error rate** on LibriSpeech test-clean.
Practical Implication
For speech pipelines, this suggests testing parallel transcript refinement as an alternative to token-by-token decoding. The key training lesson is that a connectionist temporal classification loss through the frozen output head was needed to make the model attend to audio.
Agent-Ready Context
The system connects a frozen Whisper encoder to the **26B DiffusionGemma** backbone through a projector and low-rank adapters. It trains about **42M parameters**, or 0.16% of the backbone, and reaches **6.6% word error rate** on LibriSpeech test-clean. For speech pipelines, this suggests testing parallel transcript refinement as an alternative to token-by-token decoding. The key training lesson is that a connectionist temporal classification loss through the frozen output head was needed to make the model attend to audio. The model uses roughly eight denoising steps regardless of utterance length and one adapter trained on six languages, but the paper reports evaluation only on English, Hindi, and Mandarin. The supplied results do not compare latency or accuracy with production ASR systems.
Context Map
modelaudio#generative-mediaUncertainty
The model uses roughly eight denoising steps regardless of utterance length and one adapter trained on six languages, but the paper reports evaluation only on English, Hindi, and Mandarin. The supplied results do not compare latency or accuracy with production ASR systems.