Google AI Launches TabFM for Zero-Shot Tabular Prediction

Google Research has introduced TabFM, a foundation model designed to handle tabular data through zero-shot classification and regression. By reframing tabular prediction as an in-context learning problem, TabFM allows for predictions on unseen datasets without the need for dataset-specific training, hyperparameter tuning, or manual feature engineering. The model is currently available on Hugging Face and GitHub, with plans for integration into Google BigQuery via an AI.PREDICT SQL command.

A New Paradigm for Tabular Data

For years, the field of tabular data analysis has been dominated by tree-based methods such as XGBoost, AdaBoost, and random forests. While effective, these models often require significant manual effort, including extensive hyperparameter optimization and domain-specific feature engineering, to extract reliable signals from raw data. TabFM addresses this bottleneck by applying the zero-shot logic commonly associated with large language models. Instead of updating model weights for each new dataset, TabFM processes the entire dataset as a single unified prompt, allowing it to perform predictions in a single forward pass.

Hybrid Architecture and Synthetic Training

TabFM utilizes a hybrid design that synthesizes TabPFN-style row and column attention with TabICL-style in-context learning. The architecture employs a multilayer attention module that alternates between columns and rows to capture complex feature interactions and dependencies. After this contextualization, row information is compressed into dense vectors, and a dedicated Transformer processes these embeddings to generate predictions efficiently.
Because high-quality, open-source tabular datasets are often scarce or restricted by proprietary schemas, Google trained TabFM entirely on hundreds of millions of synthetic datasets. These datasets were generated using structural causal models, which allowed the research team to incorporate a wide variety of random functions and complex feature relationships. This approach ensures the model generalizes effectively to real-world data despite being trained on synthetic examples.

Performance and Implementation

The research team evaluated TabFM using TabArena, a benchmark that calculates Elo scores based on head-to-head win rates across 38 classification and 13 regression datasets. The results indicate that TabFM consistently outperforms heavily tuned, industry-standard supervised algorithms. Two configurations are available: a plain version that runs out-of-the-box and a TabFM-Ensemble version that incorporates cross features, SVD features, and Platt scaling for classification.
Getting started with TabFM requires Python 3.11 or later and specific versions of JAX and Flax. The model is compatible with scikit-learn, allowing users to load pre-trained weights from the Hugging Face Hub and execute predictions with a standard fit and predict workflow. Whether applied to customer churn, credit risk, or house price prediction, the model is designed to provide immediate results without the traditional training cycle.