Tencent Open-Sources AngelSpec: A Unified Training Framework for Speculative Decoding

Real-world serving traffic rarely matches the averaged benchmark mixtures used in typical speculative-decoding research. To tackle this, Tencent has introduced AngelSpec, an open-source unified training framework that treats workload heterogeneity as a core design constraint by specializing structures, training data, and verification depths.

Speculative decoding operates as a lossless acceleration method where a lightweight drafter proposes future tokens, and a target model verifies them in a single forward pass. Performance depends directly on the number of accepted tokens and the round-trip duration. While high-entropy conversations benefit from shorter candidate sequences via autoregressive MTP, code and mathematical reasoning feature predictable spans that are better handled by block-parallel drafting.

Stock photo for illustration only, not from the actual event

To leverage this, AngelSpec incorporates two complementary drafters: an MTP model trained on conversational data and a block-diffusion model strengthened with code and math samples. It resolves train-inference mismatches through a shared-parameter, multi-depth scheme combined with target-model rollout from a frozen backbone.

Measured results show that at T = 0, mean acceptance increases from 52.8% to 66.4%, with deeper positions showing dramatic improvements on benchmarks like GSM8K and HumanEval. Furthermore, the DFly block-diffusion architecture—built upon DFlash with hybrid target-conditioning and a predecessor-conditioned autoregressive head—pushes performance even higher, supported by an expanded dataset of 700,000 prompts.

Source: MarkTechPost