Nokia Open-Sources AnyJev to Turn Open LLMs Into Decision Models
Nokia has open-sourced AnyJev, a training-free layer that transforms any open LLM into a calibrated decision model with transformers and vLLM backends.
Stock photo for illustration only, not from the actual event
- Nokia open-sourced AnyJev, a training-free layer for converting open-source LLMs into decision models.
- It extracts probabilities directly from next-token distribution without generating text or retraining.
- Available via PyPI under the Apache-2.0 license with transformers and vLLM backends.
Nokia has introduced AnyJev as an open-source project, serving as a training-free layer designed to turn any open large language model into a precisely calibrated decision-making model. This tool aims to help developers deploy language models into business-critical decision workflows with significantly higher reliability.
Regarding deployment, AnyJev installs seamlessly via PyPI and ships under the Apache-2.0 license. It features built-in backends for transformers and vLLM with shared-prefix scoring, ensuring fast and efficient execution suited for production environments that demand low latency.
AnyJev borrows its interface design from Jev, a System One decision model launched by TypeSafe AI in September 2026. The mechanism allows users to input a typed question and receive a decision along with a thresholdable probability. This probability is read straight from the model's next-token distribution, eliminating any need for text generation, parsing, or model training.
The absence of a training requirement is a major advantage of AnyJev, drastically reducing computational overhead. Furthermore, reading confidence scores directly from next-token outputs overcomes the common volatility where prompt option reordering alters model outputs in standard LLMs.
Nokia's research team highlights two primary flaws in conventional open-source shortcuts that restrict next tokens to option labels. First, answers can fluctuate when options are reordered. Second, the raw probabilities lack proper calibration, making them unreliable for critical threshold settings.
To overcome these challenges, AnyJev implements two distinct structural levels:
- L0 (Zero Labels): Enabled by default, applying two fixes for ordering and calibration. It incurs K prefills per decision, taking roughly 0.25 seconds per decision at batch 32 on a single H100 with K = 20.
- L1 (100 to 500 Labels): Adds temperature scaling on top of L0. Fitted values are stored as a compact JSON artifact, reshaping confidence without altering answer rankings.
Source: MarkTechPost
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment