Linkup Research Releases SPARSEUP 149M Sparse Model

Linkup Research has announced the release of SPARSEUP, a 149-million parameter open-source sparse embedding model. The model weights are now accessible on Hugging Face under an Apache 2.0 license, allowing users to load them seamlessly via Transformers or Sentence Transformers with trust_remote_code=True enabled.

While most open retrieval models rely on dense vectors mapping one vector per text, sparse models output weights distributed across a vocabulary. Each dimension corresponds to a real token, allowing vectors to fit neatly into inverted indexes while remaining human-readable and efficient at matching rare words.

Stock photo for illustration only, not from the actual event

The project was inspired by LightOn's release of DenseOn and LateOn, which provided open data, training recipes, dense models, and late-interaction models. SPARSEUP fills the missing sparse category, enabling side-by-side comparisons of all three retrieval styles using the same underlying backbone family and fine-tuning datasets.

Training originates from the LateOn-unsupervised checkpoint, which lacked an MLM head. The team grafted ModernBERT's original MLM head back onto the model and fine-tuned it using LightOn's mixture with contrastive learning exclusively. Each query incorporates 7 hard negatives from a pool of 50 alongside in-batch negatives, avoiding cross-encoder distillation entirely while training efficiently on a single H100 GPU.