GLM-5.3-Flash vs Qwen3.8-Flash-Next: AI Architecture
Z.ai and Qwen independently converge on near-identical AI model architectures, adopting a 3:1 hybrid ratio and 4-way gated residual streams.
Stock photo for illustration only, not from the actual event
- Z.ai and Qwen independently designed strikingly similar AI architectures.
- Both adopted a 3:1 hybrid ratio of linear and full attention.
- The traditional single residual stream was expanded into four parallel branches.
Artificial intelligence research labs in China have reached a remarkable milestone as two independent teams, Z.ai and Qwen, converged on virtually identical core model configurations without prior coordination, sharing design choices across attention mechanisms and optimizers.
GLM-5.3-Flash serves as the first natively multimodal model in the GLM-5 series, released openly under an MIT license on Hugging Face. Z.ai initially tested it anonymously under the moniker Ox Alpha on OpenRouter, where it quickly surged to become the most popular model of the week.
Meanwhile, Qwen3.8-Flash-Next acts as an early public preview for the upcoming architecture family. Its model card outlines a 125B main model accompanied by an additional 51B n-gram embedding table, activating 6B parameters per token with a native context window of 262,144 tokens.
Stock photo for illustration only, not from the actual event
Examining layer configurations, GLM-5.3-Flash stacks 45 total layers consisting of 34 linear-attention and 11 full-attention layers. In comparison, Qwen3.8-Flash-Next utilizes 48 layers structured in repeating blocks of 3 Gated DeltaNet layers and 1 Qwen Sparse Attention layer, perfectly landing on the same 3:1 ratio.
"Two labs, two implementations, one identical conclusion: four gated streams beat one."
AI Researchers
This architectural convergence highlights a shared industry realization regarding the limitations of traditional Transformers. By coupling linear attention with compressed retrieval indexers, both labs successfully mitigated KV cache bottlenecks, enabling higher throughput and massive context handling without sacrificing performance.
The single point of divergence lies in rotary position embeddings (RoPE) within full-attention layers. While GLM-5.3-Flash drops them entirely and relies on recurrent linear layers for positional flow, Qwen retains RoPE, noting that the NoPE variant encountered text-generation termination issues during post-training RLHF stages.
Source: MarkTechPost
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment