Can prior-data fitted networks be adapted for tabular anomaly detection?

Tabular anomaly detection has lacked a reusable backbone

Image anomaly detection was rebuilt on representations taken from models pretrained once at scale: features from generically pretrained networks, scored by simple nearest-neighbor distance, outperformed elaborate self-supervised detectors, and the same features now underlie industrial defect detection. Tabular data could not follow that path because it lacked comparably strong generic representations. Classical detectors such as Isolation Forest remain strong but start from zero on every deployment, and deep methods train one model per dataset with gains that have not proven robust across datasets. The ADBench benchmark study concluded that no unsupervised detector in its comparison was reliably better than the others.

Prior-data fitted networks (PFNs), with TabPFN chief among them, are transformers pretrained on synthetic tabular data. This paper asks whether their representations can serve as the missing generic extractor for tabular anomaly detection, and answers yes in two steps: ZEN, which changes nothing in the weights, and FOCUS, which updates them.

Two unknowns make the task harder than it looks

The detector receives a reference set describing normal behavior and must score how strongly each new sample deviates from it. Two things are unknown at once. First, the anomalies themselves: they are rare and their structure is unknown in advance, so the task cannot be posed as supervised classification. Second, the reference set may itself be contaminated, as when fraud investigators verify only a small fraction of transactions and the unverified rest enter the historical record as normal.

The paper therefore studies two settings. The main setting is the contaminated regime, where the reference set contains anomalies at an unknown rate and nothing removes them before the detector sees it. The clean regime, where the reference set is anomaly-free but unlabeled, is the complementary setting. Contamination matters because density-based detectors rate a sample by how far it sits from its nearest reference samples; an anomaly already in the reference set makes test anomalies that resemble it look normal.

ZEN scores frozen embeddings without any training

ZEN (Zero-training Embedding Neighbors) scores a test sample by its distance to its nearest neighbors in the frozen model's embedding space. The paper develops a feature-extraction procedure suited to the task and weights each reference sample by how much it can be trusted, so the score stays reliable even when the reference set is contaminated.

The readout takes the embeddings of the last four of TabPFN-3's 24 transformer blocks. Under contamination the last one to four blocks are equivalent within 0.3 AUROC and never significantly different, while every wider or deeper window is worse: reading all 24 blocks costs 2.7 AUROC points for ZEN, and single blocks improve steadily with depth from block 8 on. On clean data the picture is flatter, with the frozen readout gaining 0.1 to 0.2 points from wider windows of the last 6 to 12 blocks.

Each ZEN step adds a measurable gain

The ablation ladder starts from the plain readout: the frozen model's embedding of the data, read by kNN distance and nothing else. Under contamination the plain readout reaches 74.1 mean AUROC over the 47 datasets, next to raw-feature kNN's 73.9 and the best deep detector. The augmented context adds 0.9 points. The embedding with feature subsets adds 1.9 points. The soft cleaning adds 2.8 points, the largest single step. Together the steps take the frozen model from 74.1 to 79.7, above every baseline.

The soft cleaning is the mechanism that addresses contamination directly. Its isolation estimate scores every reference sample by how far it sits from its nearest other reference samples, reaching 76.7 AUROC when used as an anomaly score for the reference samples. Among the 5% of reference samples the pooled estimate distrusts most, 31% are anomalies, against 12% in the reference set as a whole, and the trust weights built from it average 0.70 on a reference anomaly and 1.16 on a normal sample. Pooling across representations matters: the pooled estimate beats single representations on 43 of the 47 datasets.

FOCUS fine-tunes, but adds less once ZEN is in place

FOCUS (Fine-tuned One-Class Unsupervised Scoring) builds on ZEN: a trust-weighted compactness fine-tune of TabPFN, using the reference set so that the resulting features better separate normal samples from anomalies. The only training anywhere in the pipeline is this two-epoch fine-tune.

The more of ZEN's steps are already in place, the less the fine-tune adds. Under contamination it contributes about 1.9 AUROC points on the plain readout, but only 0.3 after the soft cleaning. One reading the authors offer, which they say they have not tested directly, is that the fine-tune and the soft cleaning repair the same defect: a reference set polluted by anomalies. On clean data, where there is nothing to clean, the fine-tune keeps a significant gain of 0.6 to 1.2 AUROC points at every step.

FOCUS leads ADBench and loses less to contamination

On ADBench, ZEN reaches a higher mean AUROC than every baseline in the comparison, and FOCUS improves on it further, reaching 80.0 mean AUROC in the contaminated regime against Isolation Forest's 76.4 and raw-feature kNN's 73.9. FOCUS is tested against each of 16 baselines with a two-sided Wilcoxon signed-rank test on the 47 paired values, Holm-corrected; all 16 are significant, with the largest adjusted p-value at 0.034 against raw-feature kNN and TabPFN-PL. Against uTabPFN, on its 44 datasets, FOCUS is ahead by 13.1 AUROC points.

The contamination cost is where the comparison becomes concrete. Moving from the clean to the contaminated regime, raw-feature kNN loses 11 AUROC points on average, while FOCUS loses only 5.9. FOCUS limits the damage by finding anomalies hiding in the reference set and preventing them from hiding the test anomalies. In the clean regime the ranking narrows: FOCUS reaches 85.9 mean AUROC, with ZEN at 85.2, raw-feature kNN at 85.1, and several baselines close behind.

The approach transfers across PFN backbones

The method is not tied to a single PFN model. Applied without any change to four other tabular foundation models, using the same feature subsets, blocks, constants and protocol, ZEN lifts every backbone by roughly five to eight AUROC points, and every gain is statistically significant. TabPFN v2, TabICL and Mitra are, like TabPFN-3, prior-data fitted networks trained on synthetic priors; TabDPT is pretrained on real tables instead. For TabICL and TabDPT the results are about as strong as with TabPFN-3.

The fine-tune transfers as well. On TabPFN v2 under contamination, with the contaminated regime's constants unchanged, the fine-tune lifts the plain readout from 67.6 to 70.7 AUROC across 44 datasets, and FOCUS reaches 76.1, above ZEN's 75.9 on the same datasets. As on TabPFN-3, the fine-tune's gain is large on the plain readout and small once ZEN's steps are in place.

Where these results stop and what to prioritize

Sensitivity to constants is low: no constant sits at a sharp optimum. Under contamination, reading the last one to four blocks gives a similar result, k = 50 sits on a plateau, and every lambda between 0.25 and 2 keeps ZEN above the strongest deep baseline. For FOCUS, reducing trainable blocks to three lowers mean AUROC by 0.5 points in both regimes, and training for one or three epochs instead of two changes it by at most 0.2 points. The gain of the soft cleaning grows with the contamination rate: when 2% of the reference set are injected anomalies the cleaning adds 0.1 AUROC points, and when 20% are, it adds 2.0.

Two boundaries qualify the results. Scoring runs TabPFN forward passes for each of the B+1 = 6 feature subsets, so both methods need a GPU; on one A100 the method processes 3,500 samples per minute, slower than many classical detectors, though fewer leave-one-fold-out folds cost nothing measurable and fewer feature subsets trade marginal accuracy for a significant speedup. ADBench's standard evaluation subsamples datasets above 10,000 samples, so the largest reference set the methods have seen holds 10,000 samples, and three datasets with more than 200 features used a fixed 200-feature subset.

The practical priority follows from the contamination split rather than from the headline mean. If the reference set is believed clean, the gap between ZEN and FOCUS narrows and the fine-tune's contribution is small; if contamination is plausible, the soft cleaning is the step that carries the largest single gain, and its benefit scales with the contamination rate. The fine-tune is most worth its cost on a plain readout, before ZEN's steps are in place.

Sources used in this answer