IBM Releases Granite 4.2: Open Enterprise Models
IBM launches Granite 4.2 in three sizes featuring native reasoning and Agentic RL under Apache 2.0, trained on 15 trillion tokens.
Stock photo for illustration only, not from the actual event
- IBM released three Granite 4.2 language model sizes: 3B, 8B, and 30B under the Apache 2.0 license without licensing gates.
- The architecture features a decoder-only dense transformer supporting a 131,072-token context length, extendable to 512K.
- The 8B and 30B models run the agentic RL block, while the 3B model utilizes foundational RL and alignment.
- Training was executed using NeMo-RL and NeMo-Gym on an NVIDIA GB200 NVL72 cluster hosted by CoreWeave.
IBM has officially announced the release of its Granite 4.2 large language model family. All three language models ship under the Apache 2.0 license, meaning downloading, fine-tuning, and commercial production use carry no licensing gates.
Architecturally, Granite 4.2 is a decoder-only dense transformer rather than a hybrid or Mixture of Experts (MoE) design. Core components include Grouped Query Attention with 8 KV heads, RoPE with θ = 10,000,000, SwiGLU MLPs, RMSNorm (ε = 1e-5), untied input/output embeddings, and bfloat16 precision.
The specific dimensions across the three model sizes include:
- The 3B model utilizes 40 layers at an embedding size of 2560.
- The 8B model utilizes 40 layers at 4096.
- The 30B model scales up to 64 layers with an MLP hidden size of 32,768.
The published architecture table details a 131,072-token (128K) sequence length, while the five-phase pre-training run features a long-context phase extending up to 512K tokens, covering roughly 15 trillion tokens from scratch.
IBM's commitment to a dense transformer architecture paired with native agentic reinforcement learning for larger variants highlights a strategic approach to open-source enterprise AI. By embedding reasoning pathways directly into the model weights, developers gain advanced task-execution capabilities without the routing complexities inherent in MoE models.
Supervised fine-tuning utilizes approximately 7.2 million samples, amounting to roughly 100 billion tokens with about 65 billion trainable. The data mixture consists of 31.6% agentic and 68.4% non-agentic data, with software engineering taking up 69% of the agentic slice. Trajectories were generated across harnesses including OpenHands, SWE-agent, Terminus-2, MiniSWE, Codex, and Goose.
Stock photo for illustration only, not from the actual event
The agentic RL block runs exclusively on the 8B and 30B models, while the 3B model is limited to foundational RL and alignment. This single architectural decision accounts for the performance gap across sizes. Training was conducted via NeMo-RL and NeMo-Gym on an NVIDIA GB200 NVL72 cluster hosted by CoreWeave.
"All three Granite 4.2 language models ship under Apache 2.0, so download, fine-tuning, and commercial production use carry no licensing gate."
IBM Research
Additional supporting assets include 1 trillion tokens of synthetic code derived from IBM's CodeAlchemy pipeline and a speculative decoding layer designed for accelerated serving. Meanwhile, the Turbo CTC models feature 470 million parameters, dropping the LLM backbone entirely to use connectionist temporal classification for audio-to-text translation, achieving an RTFx throughput near 12,600 on a single H200.
Source: MarkTechPost
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment