MiniMax-Music3 Open-Weights Music Model Released

MiniMax has made waves in the artificial intelligence community by releasing MiniMax-Music3, an open-weights music generation model capable of producing full five-minute songs derived from lyrics and structured captions. Unlike research previews, the company published usable model weights, inference code, and three documented serving paths right on day one, allowing developers to deploy the technology immediately.

Under the hood, MiniMax-Music3 combines a hierarchical autoregressive stack with a continuous synthesis path. The training tokenizer employs eight layers of residual vector quantization (RVQ). The first semantic codebook holds 16,384 entries carrying core musical semantics and structure, while the remaining seven acoustic codebooks feature 1,024 entries each to encode residual details. Training optimizes the semantic layer first before jointly tuning all eight layers.

Stock photo for illustration only, not from the actual event

The Hybrid-LM approach divides the modeling task between an 8B Global LLM, which predicts the first RVQ codebook frame by frame to maintain long-range structure, and a 0.6B Local LLM handling the remaining codebooks within each frame. The model card and license indicate the Global LLM initializes from Qwen3-8B, though a MiniMax Research post references Qwen3.5-8B, leaving the exact base checkpoint somewhat unsettled.

The synthesis stage represents a particularly intriguing design choice. Instead of decoding from discrete RVQ tokens, MiniMax fuses the final hidden states of both LLMs to condition a 2.4B flow-matching module. This maps into a latent space subsequently decoded by a 123M Flow-VAE inherited from MiniMax Speech, meaning the discrete tokenizer decoder is completely bypassed during inference.

For practical usage, users input lyrics featuring dedicated section tags on individual lines—such as [Intro], [Verse], [Pre-Chorus], [Chorus], [Post-Chorus], [Bridge], [Instrumental], [Solo], and [Outro]—alongside a Structured Caption encompassing Global Metadata, Vocal Details, and Arrangement. MiniMax also provides a music-caption-rewriter agent skill to expand short descriptions into this three-part format offline.

Three documented deployment paths are currently available. SGLang-Omni serves as the reference server requiring two CUDA GPUs—GPU 0 handles Qwen3 and RVQ autoregressive generation, while GPU 1 runs flow matching and DAV decoding. The diffusers modular pipeline fits within 24 GB VRAM at full precision, dropping to approximately 22 GB with automatic CPU offload and down to 8 GB with leaf-level group offloading. Additionally, ComfyUI offers a native Text to Music template utilizing repacked FP16/INT8 weights from Comfy-Org.

Source: MarkTechPost