webAI Releases TwIL-LM: 1.7B and 3B Formal-Logic Models for Local Hardware

webAI has released a new model family named TwIL-LM, featuring 1.7B and 3B parameter versions tailored for autoformalization tasks on local hardware. Both model checkpoints are distributed under the webAI Non-Commercial License version 1.0, meaning any revenue-generating deployment requires a separate agreement directly with webAI.

The underlying training process consists of four distinct stages built on top of the base model. It starts with LoRA supervised fine-tuning on a synthetic formal-logic corpus, followed by checkpoint fusion by averaging intermediate SFT checkpoints in parameter space. This is succeeded by WiSE-FT interpolation back toward the pretrained base at a lambda value of 0.25, ending with MGPO, an entropy-weighted GRPO stage executed against a programmatic verifier. The published checkpoint sits precisely at step 2071.

The lambda value is load-bearing because only a quarter of the fine-tuned delta is retained in the final model. A sibling variant that bypassed this interpolation achieved a higher in-domain score with a macro gate of 0.515, but sacrificed roughly twelve points of held-out capability, leading webAI to withhold that specific arm from publication.

Stock photo for illustration only, not from the actual event

Efficiency highlights the model card without ambiguity, as TwIL-LM3 generates the shortest completions among all variants with 482 tokens on Track B. Consequently, it achieves the highest throughput at 32.9 answers per second, compared to just 4.2 answers per second for the 120B model.

The 1.7B variant represents a different set of trade-offs, achieving a macro-primary score of 0.361 compared to 0.185 for the unadapted base. Out-of-distribution benchmarks show mixed results, with LogicBench BQA improving to 0.590 from 0.563, while GSM8K drops to 0.380 from 0.413 and ARC-C chain-of-thought declines to 0.463 from 0.587.

Source: MarkTechPost