Accepted Prefixes Are Not All You Need: A Negative Result on PEFT-Based Block-Diffusion Drafting

Jul 15, 2026

A new preprint reports that PEFT-BD, a speculative decoding method using a LoRA-like adapter as a block-diffusion drafter, does not achieve practical speedup in language model inference. Although the approach is parameter-efficient and produces nontrivial accepted prefixes, each speculative step still requires both a full draft and verification pass through the backbone model, resulting in no compute efficiency gains. The study demonstrates that parameter efficiency alone is insufficient for speculative decoding to be effective if draft computation is not substantially cheaper than verification.

Why it matters: This negative result clarifies that compute efficiency, not just parameter efficiency, is essential for practical speculative decoding methods.

Full story at: arXiv AI/ML ↗