TabPack: Efficient Hyperparameter Ensembles for Tabular Deep Learning
TabPack trains many MLPs with sampled hyperparameters in one run and picks ensemble members on the fly, matching tuned tabular baselines out of the box — a default MacBook run beat some baselines' GPU tuning time.
**The gist** **TabPack** is an MLP ensemble for tabular deep learning that, in a **single run**, samples and trains many MLPs with different hyperparameters in parallel and selects ensemble members on the fly during training. You supply hyperparameter **ranges** instead of exact values; on medium-to-large public datasets, defaults match extensively tuned prior methods.
**Why it matters** Off the agent path, but handy if your product carries a tabular ML component: it removes the tuning loop that dominates tabular deep-learning cost. The authors report the **default configuration on a modern MacBook** finished in less time than tuning some baselines took on an **industry-grade GPU** — cheap enough to fold into a local pipeline.
**The gist** **TabPack** is an MLP ensemble for tabular deep learning that, in a **single run**, samples and trains many MLPs with different hyperparameters in parallel and selects ensemble members on the fly during training. You supply hyperparameter **ranges** instead of exact values; on medium-to-large public datasets, defaults match extensively tuned prior methods. **Why it matters** Off the agent path, but handy if your product carries a tabular ML component: it removes the tuning loop that dominates tabular deep-learning cost. The authors report the **default configuration on a modern MacBook** finished in less time than tuning some baselines took on an **industry-grade GPU** — cheap enough to fold into a local pipeline. **Watch out** The comparison set is **tuned prior deep-learning methods**; the abstract doesn't say how TabPack fares against **gradient-boosted trees**, still the default tabular baseline, or in cases that genuinely need specialized hyperparameters.