Velocity error and fluctuation energy answer different questions
The paper separates two error measures that are easy to conflate. The first, relative L2 velocity-field error, compares predicted future velocity fields with measured particle-image-velocimetry (PIV) fields over time, space, and the two in-plane components. The second, fluctuation-energy error, compares the resolved temporal fluctuation energy of those velocities, computed per cell from the two in-plane components. The authors call this quantity TKE, but they explicitly note it is not full three-component turbulent kinetic energy.
That distinction matters because a forecast can track the mean velocity field while misrepresenting how much the flow fluctuates. The study's central claim is not that one metric is wrong, but that the two can rank the same models differently. All results come from one paired simulation–measurement benchmark: flow around a NACA4418 airfoil, with 20 observed PIV frames mapped to 20 future frames.
Removing energetic fluctuations changes forecasts more than random removal
To probe what the surrogates respond to, the authors built a mean-preserving intervention on the observed input history. For each 20-frame history they computed a fluctuation-energy map, then selected the highest-energy 10% of valid observed cells. Inside that mask, each cell's velocity was replaced by its own local temporal mean, deleting the fluctuation while keeping the time average fixed. A random mask of equal area, averaged over ten deterministic draws, served as the control.
For the reference surrogate, energetic removal changed the future velocity forecast more than random removal by 0.0434 (95% trajectory-bootstrap interval [0.0393, 0.0471]) on the selection subset and 0.0431 ([0.0390, 0.0468]) on calibration. The corresponding fluctuation-energy response contrasts were 0.3508 ([0.3294, 0.3726]) and 0.3463 ([0.3259, 0.3663]). All 23 trajectory aggregates were positive on both subsets.
The contrast repeated across architectures at the 10% mask: 0.423/0.421 for FNO, 0.397/0.392 for CNO, and 0.493/0.482 for Transolver on selection/calibration. For the reference, the contrast stayed positive at 5% and 20% masks, rising from 0.285/0.284 to 0.363/0.358, which the authors describe as growth followed by partial saturation rather than a sign change.
Equal-area masks do not equalize removed fluctuation energy
The intervention's control is weaker than it first appears. Masks are matched for area, not for the amount of fluctuation energy they delete. Because the energetic mask is built from the input history itself, it generally removes more fluctuation energy than a random mask of the same size. A larger forecast change can therefore reflect a larger input perturbation rather than a special sensitivity to physically important regions.
The authors state this limit directly: the energetic-minus-random contrast measures sensitivity, not independent evidence of physical importance. An energy-matched control or an analysis normalized by removed input energy would be needed to separate the two explanations, and neither is reported. The masked histories are also diagnostic inputs, not solutions of the governing equations, and their hard boundaries may interact with spectral operators. No future ground truth exists for these modified histories.
The CNO ranks better on velocity but worse on fluctuation energy
The clearest disagreement comes from comparing separately trained models against the reference on identical windows. On the selection subset, the CNO's velocity-field error was lower than the reference by 0.0189 (95% interval [0.0265, 0.0117] lower), while its fluctuation-energy error was higher by 0.1196 ([0.1016, 0.1369]). Calibration gave the same pattern: 0.0191 lower field error and 0.1232 higher fluctuation-energy error. The opposite ranking held for 18 of 23 trajectory aggregates on each subset.
FNO is a useful contrast rather than a second example. Its velocity-field advantage was larger and robust, but its small positive fluctuation-energy difference had an interval spanning zero, so the authors do not count it as a ranking inversion. Transolver was worse on both metrics. A horizon-conditioned decoder variant supplied a smaller within-lineage replication, with lower field error and higher fluctuation-energy error whose intervals excluded zero in opposite directions.
Attenuation lowers benchmark error while deepening an energy deficit
An output-side stress test sharpens the point. The authors rescaled predicted velocity fluctuations by fixed factors of 0.975 and 0.935 while preserving the temporal mean exactly, without retraining. On selection, velocity-field error fell from 0.120958 to 0.119692 and fluctuation-energy error from 0.533389 to 0.528463; calibration showed the same direction.
Yet the reference model was already under-energetic, with a domain-summed predicted-to-measured fluctuation-energy ratio of about 0.815, and attenuation pushed that ratio down to about 0.756. Error in the measured top-energy decile rose by 3.9% and 3.7%, while error over the remaining 90% fell by about 12.8%, and spatial fluctuation-energy correlation was essentially unchanged. The two diagnostics need not rank predictions identically because the benchmark error averages separately normalized per-sample residuals, whereas the energy ratio pools fluctuation energy over the domain.
The attenuation gains were selected using both analysis subsets, so the authors treat this as a diagnostic stress test, not a held-out correction. Under five-fold trajectory-wise cross-fitting, the held-out energy mismatch worsened by 0.0390 (95% interval [0.0218, 0.0534]), while the fluctuation-energy error change was not resolved away from zero.
The evidence covers 23 trajectories, not the full release
The two analysis subsets, labeled selection and calibration, contain 184 and 768 windows. They are window-disjoint but not trajectory- or condition-disjoint: both draw from the same 23 experimental trajectories and operating-condition support, so calibration is a within-trajectory replication rather than an unseen holdout. The paper's conclusions concern those 23 trajectories, not all 100 released trajectories, and the four tested surrogates constrain how far the architecture comparison generalizes.
The benchmark itself spans angles of attack from 0° to 20° and Reynolds numbers from 2968 to 27975, with fields released at 64×128 and scored after 2× spatial subsampling at 32×64. Statistical intervals use 5,000 percentile bootstrap replicates over complete trajectory IDs. The authors also note that disagreement between metrics shows a scalar benchmark score gives an incomplete account of measured flow statistics; it does not by itself determine which model would perform better in a downstream physical application.
Report physical diagnostics alongside aggregate scores
The paper's practical priority follows from the CNO comparison rather than from the intervention alone. When one model has lower velocity-field error but higher fluctuation-energy error on both subsets, a leaderboard built only on the aggregate score would rank it ahead while hiding a larger deficit in resolved fluctuation energy. The authors therefore argue for reporting complementary physical diagnostics, including the domain-summed fluctuation-energy ratio, spatial fluctuation-energy correlation, pooled squared error, and error inside the measured top-energy decile, next to the standard prediction scores.
The intervention results add a second, weaker signal: forecasts respond more to removing fluctuations from the most energetic observed cells than to equal-area random removal, consistently across four architectures and three mask fractions. Because removed energy was not matched, that contrast should be read as sensitivity under the tested setting, not as proof that models recognize physically important regions. The paper's defensible conclusion is narrower and more useful: a better score is not evidence of better physics, and the CNO case shows the two can point in opposite directions on the same benchmark.
Sources used in this answer
Do Better Scores Mean Better Physics? Physics-Grounded Explanations for Sim2Real Neural Operators
On a NACA4418 paired PIV/CFD benchmark, a CNO had lower velocity-field error but substantially higher two-component fluctuation-energy error than the reference on both analysis subsets, and an attenuation stress test lowered benchmark error while deepening an energy deficit.