EcoSpec: Cost-Aware Speculative Decoding for Mixture-of-Experts Models

Jul 15, 2026

EcoSpec is a cost-aware speculative decoding framework designed to improve inference efficiency in large-scale Mixture-of-Experts (MoE) language models. By incorporating predicted expert activation costs into draft token selection, EcoSpec encourages reuse of already-loaded experts during speculative decoding. Evaluations on models such as DeepSeek-V3.1 (671B), Qwen3-235B-A22B, and GPT-OSS-120B show that EcoSpec reduces expert activation footprints and achieves up to 1.62× speedup in end-to-end decoding, without modifying verification rules.

Why it matters: This work offers a practical advance for accelerating inference in large MoE language models by optimizing draft selection for expert reuse, directly improving decoding speed and efficiency.

Full story at: arXiv Computation and Language ↗