Cursor Open-Sources Mixture-of-Kittens (MoK) Megakernel for GB300 NVL72 Racks
Cursor releases MoK, a deterministic Mixture-of-Experts training megakernel designed for NVIDIA Blackwell architecture, boosting training speeds up to 2.37x.
Stock photo for illustration only, not from the actual event
- Cursor open-sources Mixture-of-Kittens (MoK) under the Apache-2.0 license.
- Requires NVIDIA Blackwell SM100 or SM103 GPUs (GB200/GB300 NVL72 racks).
- Delivers up to 2.37x faster MXFP8 forward pass performance.
- Minimizes CPU-GPU synchronization to resolve communication bottlenecks.
Cursor has officially open-sourced Mixture-of-Kittens (MoK), a deterministic Mixture-of-Experts (MoE) training megakernel, available on GitHub under the Apache-2.0 license. The software is engineered to squeeze maximum performance out of cutting-edge hardware infrastructure.
However, the hardware requirements for MoK set a high barrier to entry. It demands NVIDIA Blackwell SM100 or SM103 GPUs, translating to GB200 NVL72 or GB300 NVL72 rack ownership or rental. Furthermore, it requires Python 3.12+, PyTorch 2.10+, and CUDA toolkit 13.0+, with inter-GPU buffers relying heavily on PyTorch symmetric memory.
This hardware threshold restricts realistic adopters to organizations with substantial capacity, such as frontier labs, funded model startups, GPU neoclouds, and national computing centers. Single-node teams and standard 8-GPU shops are left out of the equation.
The introduction of this specialized megakernel highlights a shifting paradigm in large-scale AI training, where communication overhead within a rack often supersedes raw compute limits. By minimizing reliance on integrated Grace CPUs inside massive 72-GPU NVLink domains, MoK addresses the critical synchronization bottleneck that traditionally plagues massive MoE model scaling.
Applications remain narrow yet high-value, encompassing pretraining and post-training of DeepSeek-V3-style MoE models, on-policy RL post-training, and internal ablations where determinism is critical. Relevant sectors span AI model development, cloud GPU infrastructure, code-generation tooling, and quantitative research.
Stock photo for illustration only, not from the actual event
In production environments, communication historically served as the primary limiting factor, with the MoE layer consuming over half of end-to-end training time. Transitioning to GB300 NVL72 systems alters this dynamic by enclosing 72 GPUs inside a single NVLink domain for fine-grained overlap, though the slower integrated Grace CPUs still necessitate aggressive minimization of CPU-GPU synchronization.
Built as a fully deterministic megakernel, MoK supports BF16 and MXFP8 precision modes, scheduling via Blackwell's Cluster Launch Control to prevent inter-rack RDMA serialization. Layer benchmarks on a single NVL72 rack at EP degree 64 demonstrated that MoK achieves up to 2.37x faster performance for MXFP8 forward passes compared to the fastest baseline, while end-to-end token throughput per GPU rose from 760.9 to 1,070.2.
Source: MarkTechPost
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment