TabPFN with zero training beats tuned XGBoost on 14 out of 14 tables

TabPFN and TabICL vs. tuned XGBoost: the model that doesn't train won 14/14

The author tested TabPFN and TabICL against tuned XGBoost on 14 datasets from the Grinsztajn benchmark. Both models skip gradient training on the target table, using training rows as context in a single forward pass, and won all 14 matchups. The advantage holds up to 32,000 rows. Inference latency ranges from 0.6 to 6 seconds per row, with wider tables costing more. The post also notes that the most-cited TabPFN version now requires an account to download. The XGBoost tuning budget and hyperparameter search space are not detailed in the article.