Thinking Machines Lab Releases Inkling-Small: A 276B Open Weights Multimodal MoE Model
A compact yet fully multimodal model designed to run efficiently on accessible hardware configurations.
Stock photo for illustration only, not from the actual event
- Inkling-Small features 276B total parameters with 12B active parameters per token.
- The NVFP4 checkpoint reduces VRAM requirements to 180 GB, fitting on a single B300 or two H200s.
- Supports multiple modalities including text, images, and 16 kHz WAV audio inputs.
- Available now on Tinker and compatible with popular runtimes like vLLM and SGLang.
Thinking Machines Lab has officially launched Inkling-Small, an open-weights multimodal Mixture of Experts (MoE) model packing a total of 276 billion parameters while activating 12 billion parameters per token. Designed to move large-scale AI architectures out of exclusive frontier-lab territories, this release enables startups, mid-size enterprises, and regulated sectors to self-host or serve a powerful multimodal model without requiring massive new infrastructure investments.
Addressing hardware constraints directly, the model's standard BF16 checkpoint requires at least 600 GB of aggregated VRAM, a capacity satisfied by four NVIDIA B300 GPUs or eight NVIDIA H200 GPUs. However, the NVFP4 checkpoint dramatically lowers this entry barrier to 180 GB. This allows the model to execute W4A4 workloads on a single B300 GPU with SM100+ architecture or W4A16 on two H200 units. Supported serving runtimes include SGLang, vLLM, TokenSpeed, Unsloth, and Hugging Face.
Stock photo for illustration only, not from the actual event
Architecturally, Inkling-Small is built as a 42-layer decoder-only transformer utilizing a sparse MoE feed-forward backbone. Every incoming token routes dynamically through 6 out of 256 individual experts, alongside 2 shared experts that remain constantly active. Attention mechanisms combine local and global layers natively. The model operates encoder-free, handling images split into 40x40-pixel patches processed via a four-layer hMLP, and representing audio as dMel spectrograms. Both modalities feed into a lightweight embedding layer to process jointly alongside text tokens.
Inkling-Small began its training cycle following its larger predecessor, allowing researchers to refine the pre-training data mix and machine learning recipe. The team post-trained an earlier preview checkpoint using on-policy distillation with the larger Inkling model acting as the teacher, followed by two weeks of scaling agentic coding reinforcement learning. While benchmark metrics such as SimpleQA Verified drop to 20.6% and Tau 3 Banking to 15.5%, multimodal capabilities remain remarkably competitive at a fraction of the cost, scoring 74.0% on MMMU Pro and 90.1% on VoiceBench.
The introduction of Inkling-Small highlights a broader industry shift toward efficiency and accessibility in generative AI. By optimizing quantization down to 180 GB via NVFP4, Thinking Machines Lab bridges the gap between massive frontier models and operational deployment realities for enterprises and regulated sectors that demand data privacy and local hosting capabilities.
Regarding calibration and safety, Inkling-Small achieves a Brier Index of 61.3 ± 0.46 on ForecastBench without search, outperforming its larger counterpart. Safety evaluations report 98.4% on StrongREJECT and 96.9% on FORTRESS benign. Thinking Machines Lab notes that the model introduces no material uplift risks beyond the existing open-weight ecosystem, recommending standard downstream moderation like Llama Guard for consumer-facing deployments. Both models are currently accessible on Tinker with limited-time discounts.
Source: MarkTechPost
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment