Knowledgator Releases GLiFormer 575M Parameter Encoder
Knowledgator introduces GLiFormer, a 575M-parameter encoder hitting 91.10 F1 on nested JSON extraction without token generation.
Stock photo for illustration only, not from the actual event
- Knowledgator releases GLiFormer, a 575M-parameter encoder model.
- Achieves 91.10 F1 score on nested JSON extraction without generating tokens.
- Outperforms Gemma-4-E4B with approximately 14x fewer parameters in Large size.
- Deployable today via pip under Apache 2.0 on both CPU and GPU.
Knowledgator has officially released GLiFormer, a 575-million-parameter encoder model engineered to handle data extraction tasks without relying on the token-by-token text generation typically used by large language models (LLMs). Traditional extraction pipelines often chain multiple distinct models together to tag entities, classify documents, and rebuild records. The research team behind GLiFormer argues that these tasks fundamentally share a single core operation: encoding the source, representing the requested concepts, and scoring their compatibility.
Building upon the architecture of GLiNER, GLiFormer generalizes label matching through a concept called an 'anchor'. An anchor serves as the object against which each runtime label is scored, acting as a group vector for classification, an entity pair for relations, or a record slot. The source text is encoded precisely once, after which multiple schemas for the same document execute as task-local groups over that shared representation.
For Named Entity Recognition (NER) tasks, the model's head scores the start, end, and inside evidence for every token and label pair. Independent sigmoid outputs allow nested mentions and shared boundaries to coexist seamlessly. Furthermore, because extracted values correspond directly to source spans, the model is strictly prevented from inventing value text that is absent from the input.
Both v1 checkpoints utilize the gliformer-layout model configuration featuring five distinct heads: NER, classification, joint relations, multilevel structuring, and embeddings. Each configuration establishes a 12-word maximum span width along with 100 record anchors. The base variant, GLiFormer-base, is initialized from a DeBERTa backbone that underwent further pre-training on 100 billion tokens, utilizing 1,357,671 examples for multitask training and 372,090 for task-focused post-training.
"Extraction stacks often chain separate models. One tags entities, another classifies documents, and a third rebuilds records. The research team argues these tasks share one core operation."
By replacing token generation with an encoder-based compatibility scoring mechanism, GLiFormer bypasses the primary computational bottleneck associated with LLM-based structured outputs. While generative models must sequentially output field names and punctuation token by token, encoder architectures process the document holistically. This structural shift not only accelerates inference speeds significantly but also minimizes the risk of hallucinations since values are strictly constrained to existing text spans.
Stock photo for illustration only, not from the actual event
In benchmark performance evaluations conducted by Knowledgator, GLiFormer-base was timed across 40 structuring documents with a batch size of 1. The median latency was recorded at 69 milliseconds on an NVIDIA RTX PRO 6000 Blackwell GPU running in FP16 precision, and 547 milliseconds on an 8-thread AMD EPYC 9B45 CPU running in FP32. The headline claim of being up to 95.8x faster represents an analytical estimate compared to measured LLM runs, assuming a prefill rate of 2,000 input tokens per second and a generation rate of 60 output tokens per second, while excluding network delays and queuing.
Both checkpoints are available for immediate deployment under the Apache 2.0 license, installable via pip install gliformer, and capable of executing smoothly on standard CPU or GPU hardware. Developers and researchers can access comprehensive pre-trained model documentation, model weights, and the GitHub repository directly through Knowledgator's release channels.
Source: MarkTechPost
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment