Liquid AI Releases LFM2.5-DSpark Draft Models
Liquid AI launches LFM2.5-DSpark draft models, delivering up to 3.18x faster decoding speeds on hardware like H100 and M4 Max.
Stock photo for illustration only, not from the actual event
- Liquid AI releases LFM2.5-DSpark draft models for speculative decoding
- Delivers up to 3.18x faster decoding without changing model outputs
- Draft models feature around 300M parameters with shared target vocabulary
- DSpark architecture integrates DFlash, L-Sequential Head, and verifier
Liquid AI has rolled out the LFM2.5-DSpark draft models to enhance speculative decoding workflows, aiming to deliver up to 3.18x faster decoding speeds while preserving the original model outputs. The model weights are shipped as Safetensors and GGUF files, requiring users who self-host to build SGLang or llama.cpp with DSpark support tailored for LFM2 targets.
Regarding the architecture, each LFM2.5 draft model contains roughly 300 million parameters: specifically 295.7M parameters for the 1.2B-Instruct target and 327.7M parameters for the 2.6B and 8B-A1B targets. The backbone consists of 5 full-attention layers with a hidden_size of 2048, intermediate_size of 6144, GQA at 32 heads over 8 KV heads, and a block size of 9. The draft model ships without standalone vocabulary weights, tying both embedding and language model heads directly from the target during load time. The 2.6B draft repository size stands at 655 MB in BF16 format, representing the actual memory overhead added.
Liquid AI's DSpark framework combines three core components into its pipeline:
Liquid AI benchmarked performance on a single H100 GPU using BF16 via SGLang, and on an M4 Max MacBook Pro using llama.cpp with Metal and FP16 GGUF weights. Both setups utilized a block size of 9, a batch size of 1, and a temperature of 0 across benchmark suites including MATH500, HumanEval, MBPP, GSM8K, and MT-Bench. Speedups directly track token acceptance rates, which correspond to output predictability. For instance, the LFM2.5-8B-A1B model accepts 8.27 out of 10 tokens per step on MATH500, but only 4.02 on GSM8K, causing the speedup to fluctuate from 3.18x down to 1.29x on the same GPU.
"LFM2.5-8B-A1B accepts 8.27 of 10 tokens per step on MATH500 and only 4.02 on GSM8K, so the same model swings from 3.18x to 1.29x on the same GPU."
Liquid AI
Stock photo for illustration only, not from the actual event
Context and Analysis: Speculative decoding is a key acceleration paradigm in modern machine learning infrastructure designed to overcome the autoregressive decoding bottleneck. By leveraging smaller secondary models to propose token sequences that are then verified in parallel by a larger backbone model, inference throughput can be significantly improved. Liquid AI's introduction of DSpark optimizes this mechanism further for the LFM2 family, particularly benefiting agentic workflows requiring multi-step reasoning before tool executions.
An important caveat emerges on Apple silicon architecture: the Mixture of Experts (MoE) variant LFM2.5-8B-A1B achieves only a 1.18x average speedup on the M4 Max chip. Liquid AI attributes this performance limitation to the current MoE implementation within llama.cpp's Metal backend, noting that verifying multiple tokens activates more experts and increases weight traffic compared to a standard single decoding step.
Stock photo for illustration only, not from the actual event
Despite hardware-specific caveats, throughput gains concentrate heavily in workloads where users wait through prolonged reasoning phases prior to tool invocation. Across multi-tool function-calling scenarios, Liquid AI reports that DSpark cuts latency by 57% on average for the LFM2.5-2.6B model, proving valuable for autonomous agents that plan, execute, and re-plan multiple times per user turn.
Source: MarkTechPost
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment