Nums AI Releases Causilo Tabular Model Topping TabArena
Nums AI launches Causilo, an in-context learning tabular foundation model pretrained on synthetic data, achieving a TabArena Elo of 1794.
Stock photo for illustration only, not from the actual event
- Nums AI has released Causilo version 1.0.1, a new tabular foundation model.
- It achieves a top overall Elo score of 1794 on TabArena across 51 datasets and 816 Full splits.
- Built on in-context learning, calling fit does not update pretrained model weights.
- Compatible with Python 3.10 through 3.12 and PyTorch 2.13 or newer on CUDA or CPU.
Nums AI has officially released Causilo, a tabular foundation model designed to top the TabArena leaderboard among single models. The model is currently available for research and evaluation purposes using CUDA or CPU hardware, while commercial, production, and hosted API deployments require a separate commercial license obtained directly from Nums AI.
From an architectural standpoint, Causilo relies on an in-context learning paradigm. Executing the fit method does not alter its pretrained weights; instead, it stores the training rows as contextual information and predicts query rows within a single forward pass. Nums AI pretrained the model exclusively on synthetic data, deliberately omitting any TabArena datasets from its training pipeline.
Stock photo for illustration only, not from the actual event
The model accepts NumPy arrays and pandas DataFrames as inputs, seamlessly handling categorical features and missing values. Classification tasks support up to 10 classes, while regression tasks return mean predictions by default, with version 1.0.1 introducing median and quantile outputs derived from 999 native quantiles.
Nums AI structures the neural network into three core phases: refinement, compression, and in-context learning. Features are organized into sets of three, where each value is embedded using 16 learned sine and cosine frequencies, and missing values receive a dedicated learned vector. Two column stages subsequently summarize each feature group utilizing 128 latent slots.
By employing cross-attention instead of full self-attention between column stages, Causilo maintains computational costs that scale linearly with the number of features, effectively mitigating the scalability hurdles commonly faced by traditional tabular deep learning architectures.
To validate its performance, Nums AI utilized the official TabArena evaluation pipeline comprising 51 datasets, 816 Full splits, 8 estimators, and a random seed of 42. An independent TabArena maintainer re-executed the complete evaluation, confirming the identical overall Elo score of 1794. Furthermore, on ScoringBench across 101 datasets, Causilo ranked first in CRPS, R², and RMSE metrics.
Stock photo for illustration only, not from the actual event
Developers can install the package via pip install causilo and utilize CausiloClassifier or CausiloRegressor classes for machine learning tasks. Users can also explore the live Hugging Face demo Space or inspect the repository on GitHub and download model weights directly.
Source: MarkTechPost
Found something wrong in this article? Report an issue with this article
Comments
Leave a Comment