TabPFN vs XGBoost for Structured Data Predictions

TabPFN and XGBoost are both excellent models for structured data - but they are excellent in different regimes. The peer-reviewed evidence is clear on where each one wins.

TabPFN vs XGBoost on single flat tables: what the benchmarks show

TabPFN (PriorLabs, published in Nature 2024) is a transformer-based foundation model pre-trained on millions of synthetic tabular datasets. It uses in-context learning: you pass your data as context, and the model makes predictions immediately - no training, no hyperparameter tuning. On single-table benchmarks with up to 10,000 rows, TabPFN consistently matches or beats carefully tuned XGBoost in approximately 2.8 seconds.

XGBoost (and LightGBM) is the reigning champion for larger structured datasets. It scales efficiently to millions of rows, offers fast inference, and with expert hyperparameter tuning and feature engineering, it remains the top performer on large tabular datasets and Kaggle competitions.

When to choose TabPFN

TabPFN is the right choice when your data fits in a single table and is relatively small. Specifically:

• Small datasets (under 10,000 rows). TabPFN frequently matches or outperforms tuned XGBoost, with no effort spent on hyperparameter optimization. For rapid prototyping and quick baselines, it is unmatched.
• Zero-tuning scenarios. If you want a strong prediction in seconds without touching a single hyperparameter, TabPFN delivers. This is genuinely useful for exploratory analysis, initial feasibility checks, and quick comparisons across datasets.
• Uncertainty quantification. TabPFN natively provides prediction intervals via its Bayesian posterior approximation. For applications where calibrated uncertainty matters (medical diagnosis, risk assessment), this is a meaningful advantage over XGBoost, which requires separate implementations.
• Noisy or messy data. TabPFN handles missing values, outliers, and uninformative features robustly, often degrading more gracefully than tree-based models when data quality is poor.

When to choose XGBoost

XGBoost is the right choice when your data is large, when you need production-grade inference speed, or when expert tuning is feasible:

• Large datasets (50,000+ rows). XGBoost scales efficiently to millions of rows. TabPFN hits memory limitations and performance degradation beyond approximately 10,000-50,000 samples, depending on the version.
• Production latency requirements. XGBoost offers extremely fast inference (sub-millisecond per sample) with compact model files (megabytes, not gigabytes). For real-time serving at scale, it remains the practical standard.
• Expert feature engineering available. When you have domain experts who know which features matter and can invest weeks in crafting them, XGBoost on well-engineered features is highly competitive. The combination of domain knowledge and gradient boosting is powerful.
• Interpretability is critical. XGBoost's native feature importance scores and SHAP integration provide clear explanations of model decisions, which matters in regulated industries like banking and insurance.

The question both models cannot answer: what about multi-table data?

Every TabPFN-vs-XGBoost comparison assumes the same thing: your data is a single flat table. One row per entity, one column per feature. For Kaggle competitions and academic benchmarks, this assumption holds. For enterprise prediction tasks, it almost never does.

A typical enterprise prediction task - customer churn, fraud detection, lead scoring, demand forecasting, recommendation - requires data from 5 to 50 connected tables. Customers, orders, products, reviews, support tickets, interactions, payments, sessions. These tables are connected by foreign keys in a relational database. The predictive signal lives not just in individual tables, but in the relationships between them.

To use TabPFN or XGBoost on this data, someone must first flatten it: write SQL joins to combine tables, compute aggregations (average order value, support ticket count, days since last purchase), and collapse everything into one row per entity. This flattening step is not just tedious - it creates a hard accuracy ceiling that no amount of model sophistication can overcome.

Think about what happens when you flatten a relational database into one table. A customer who is connected to orders, which are connected to products, which are connected to reviews from other customers, who have their own purchase and churn patterns - that entire web of 3rd-degree and 4th-degree connections collapses into a single row with a few aggregate columns: order_count = 23, avg_order_value = $142. The rich graph of relationships - the connections that actually predict whether this customer will churn, commit fraud, or convert - is gone. Permanently. No model trained on that flattened row can recover what the data no longer contains. This is not a modeling limitation. It is an information limitation. Flattening does not create a penalty that a better algorithm can overcome - it creates a ceiling.

This is not a theoretical concern. On the RelBench benchmark (7 databases, 30 prediction tasks, 103 million rows across 51 tables), models that read flattened single tables score 62.44 AUROC. Models that read the relational structure natively score 76.71 AUROC zero-shot. The 14+ point gap is the cost of flattening - the information that TabPFN and XGBoost never see because it was destroyed before they received the data.

KumoRFM 2.0: the model that does both

KumoRFM 2.0 is a relational foundation model built by Kumo.ai. Unlike TabPFN (which reads one flat table) or XGBoost (which reads one flat table), KumoRFM reads multiple relational tables connected by foreign keys and discovers predictive patterns across the full relational graph using a graph transformer architecture.

But KumoRFM 2.0 is not just a relational model. It is a superset. It supports both single-table and multi-table prediction tasks:

• On single-table tasks: KumoRFM 2.0 is competitive with TabPFN and XGBoost. It provides zero-shot predictions on flat tables with no feature engineering, similar to TabPFN's approach.
• On multi-table relational tasks: KumoRFM 2.0 dramatically outperforms both. It reads 5-50 connected tables directly, discovers multi-hop predictive patterns (2-hop, 3-hop, 4+ hop signals across table boundaries), captures temporal sequences across tables, and detects graph-structural patterns - all automatically, with zero feature engineering.

This means enterprise teams do not need to choose between a small-data model (TabPFN) and a large-data model (XGBoost). They get one foundation model that covers both regimes - plus the relational dimension that neither TabPFN nor XGBoost can touch.

Why pre-training matters: the second gap flat-table models cannot close

Even if you could somehow flatten your relational data perfectly - capturing every aggregation, every temporal window, every cross-table metric - you would still face a second, fundamental disadvantage. The model itself has never seen relational patterns before.

TabPFN is pre-trained on millions of synthetic single-table datasets. It has learned the statistical patterns common to flat tabular data: feature correlations, nonlinear decision boundaries, class distributions. This is why it excels on single-table benchmarks. But it has never encountered a multi-hop relationship, a cross-table temporal sequence, or a graph-structural pattern - because those do not exist in single-table data.

XGBoost is not pre-trained at all. It learns from scratch on whatever features a data scientist provides. It is powerful on the features it receives, but it has no prior knowledge of any data structure.

KumoRFM is pre-trained on tens of thousands of diverse real relational datasets spanning different industries, schemas, and entity types. It has already learned what relational patterns look like: how multi-hop connections carry predictive signal, how temporal patterns propagate across table boundaries, how graph-structural properties predict entity behavior. When KumoRFM encounters your enterprise database, it recognizes relational patterns it has seen across thousands of prior databases. It does not search for patterns from scratch - it recognizes them.

Enterprise benchmarks: SAP SALT

The SAP SALT benchmark is an enterprise-grade evaluation where real business analysts and data scientists attempt prediction tasks on SAP enterprise data - production-quality databases with multiple related tables. Unlike academic single-table benchmarks, SAP SALT reflects how enterprise data actually looks: multiple connected tables, complex relationships, and real business outcomes to predict.

The 16 percentage point gap between KumoRFM (91%) and PhD+XGBoost (75%) is not marginal. In enterprise terms, that difference translates to millions of dollars in caught fraud, retained customers, or converted leads. And KumoRFM achieves it with zero feature engineering - no SQL joins, no aggregation pipelines, no weeks of data scientist labor.

Research benchmarks: RelBench

RelBench is an academic benchmark designed specifically to test models on relational data: 7 real-world databases, 30 prediction tasks, 103 million rows across 51 tables. It is the only major benchmark that preserves the relational structure of the data instead of pre-flattening it.

AUROC (Area Under the Receiver Operating Characteristic curve) measures how well a model distinguishes between positive and negative outcomes. An AUROC of 50 means random guessing. An AUROC of 100 means perfect prediction. Moving from 62 to 77 AUROC is a major improvement - it means the model correctly ranks a true positive above a true negative 77% of the time instead of 62%. For fraud detection, that difference means catching significantly more fraud with fewer false alarms.

The key result: even the best flat-table approaches (XGBoost/LightGBM with weeks of expert feature engineering) score in the low 60s on relational data. KumoRFM zero-shot, with no feature engineering at all, scores 76.71. Fine-tuned, it reaches 81.14. The gap is not about better gradient boosting or a better transformer - it is about what data the model can see.

A concrete example: churn prediction with multi-table data

Consider a SaaS company predicting 90-day customer churn. The data lives across 6 tables: customers, subscriptions, product_usage, support_tickets, invoices, and feature_requests.

The strongest churn predictor in this data is a 4-hop pattern: a customer's churn risk spikes when other customers who use the same product features, filed similar support tickets, and have similar usage trajectories started churning recently. This signal requires traversing Customer → Product_usage → Features → Other_customers_using_same_features → Their churn outcomes.

PREDICT churn_90d
FOR EACH customers.customer_id
WHERE customers.contract_type = 'enterprise'

The decision framework: which model should you use?

The right model depends on two questions: what is your data structure, and what is your dataset size?

Why this comparison is usually incomplete

Most TabPFN-vs-XGBoost comparisons focus exclusively on single-table benchmarks: Kaggle datasets, UCI datasets, synthetic classification tasks. On these benchmarks, the conclusion is clear and correct: TabPFN wins on small data, XGBoost wins on large data.

But these benchmarks share a structural assumption that does not hold in enterprise settings: the data has already been flattened into a single table. Someone has already done the SQL joins, the feature engineering, the aggregation. The benchmark measures model performance after the hardest and most lossy step has already happened.

For enterprise ML teams, the real question is not “which model is best on a flat table?” It is “which approach gives the most accurate predictions on my actual data - which lives in a relational database with multiple connected tables?” When you ask that question, the answer changes fundamentally. The choice is no longer between TabPFN and XGBoost. It is between flattening your data (and losing 15-20+ AUROC points of signal) or using a model that reads the relational structure directly.