Liquid AI Releases LFM2.5-Encoder-230M and LFM2.5-Encoder-350M for Fast 8K CPU Context
Liquid AI introduces new bidirectional encoders built on the LFM2.5 architecture, maintaining high speeds across an 8K context window entirely on CPUs.
Stock photo for illustration only, not from the actual event
- Liquid AI launched two encoder models sized at 230M and 350M parameters.
- Engineered to process long 8,000-token inputs efficiently on CPU-only hardware.
- The 350M variant achieved a 17-task mean score of 81.02, ranking fourth overall.
- Features grouped-query attention and gated short-convolution blocks supporting 15 languages.
Encoders operate underneath classifiers, intent routers, safety filters, and PII detectors, running continuously and handling increasingly lengthy inputs usually without a GPU. While BERT established this category and ModernBERT later advanced its speed and accuracy, Liquid AI argues that its LFM2 architecture scales its computational cost much more slowly as input lengths increase.
These encoders are not built from scratch; instead, they initialize from LFM2.5-230M and LFM2.5-350M decoder backbones, modified through three structural changes. The architecture interleaves gated short-convolution blocks with grouped-query attention, matching the design in the LFM2 technical report. Both checkpoints utilize a hidden size of 1024, a 65,536-token vocabulary, support 15 languages, and are released under the LFM Open License v1.0.
Stock photo for illustration only, not from the actual event
Liquid AI evaluated 14 models across 17 tasks sourced from GLUE, SuperGLUE, and multilingual classification benchmarks. LFM2.5-Encoder-350M secured a 17-task mean score of 81.02 (±1.00), placing fourth in the rankings. The three models ahead of it are larger: XLM-R XL at 3.5B (83.06), ModernBERT-large at 395M (81.68), and XLM-R large at 560M (81.34), with the top model being nearly 10 times its size.
Meanwhile, LFM2.5-Encoder-230M posted a score of 79.29 (±1.02) to rank sixth, outperforming ModernBERT-base at 78.19 as well as every EuroBERT model in the evaluation table, including EuroBERT-610M (75.87) and EuroBERT-2.1B (72.19). Both new encoders also surpassed Liquid AI's retrieval models, LFM2.5-ColBERT-350M (76.18) and LFM2.5-Embedding-350M (75.68).
Building a dedicated general-purpose encoder rather than relying solely on existing retrieval siblings highlights Liquid AI's strategy to optimize performance for extensive text inputs on standard hardware, which is critical for edge deployments and regulated environments where cloud round-trips are impractical.
Source: MarkTechPost
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment