Why does behavioral cloning keep improving as models overfit their datasets?

Four mysteries that real robot data keeps producing

Behavioral cloning (BC) trains a policy to copy actions from a dataset of demonstrations. It is the workhorse of large-scale robot control, yet practitioners keep reporting results that contradict standard machine-learning intuition. The paper collects four of these: performance often rises as the model overfits more, and a smaller dataset can beat a larger one; partially open-loop control with action chunking can substantially outperform fully closed-loop control even without inference delay; human-data BC often needs much larger networks than synthetic-data BC, even on state-based tasks; and input features strongly affect performance even when data is plentiful.

The obstacle is that these observations come from real human demonstrations, where experiments are costly, not perfectly reproducible, and hard to factor apart. Existing simulated benchmarks with scripted or reinforcement-learning-trained policies offer control but tend not to reproduce the phenomena. The authors attribute that gap to structural differences: previous scripted policies are often less temporally correlated, less multi-modal, and diversified mainly with Gaussian noise, and their action statistics tend to pile up at the action limits (−1 or +1) or look overly uniform, unlike human demonstrations.

Scripted policies built to look like human demonstrations

OCBench's central design choice is to generate data with scripted policies that imitate three properties the authors ascribe to human demonstrations: trajectories that are very narrowly distributed rather than Gaussian-augmented, highly non-Markovian and temporally correlated, and semantically diverse in how objects are grasped and manipulated. The data-generation backbone is randomized piecewise C1-continuous Hermite splines with Catmull-Rom tangent matching. For each task, the script samples control points for semantic phases such as grasp, lift, and transport, then interpolates smoothly between them.

Diversity comes from randomizing waypoints, grasping angles and directions, contact points, motion speeds, gripper rolls and pitches, trajectory curvatures, and high-level task-solving strategies. The scripted policy is also allowed to make mistakes and recover from them. The paper reports that OCBench's action statistics resemble actual human demonstrations more closely than previous scripted or RL-trained policies, which tend to saturate the action range or produce overly uniform distributions.

The benchmark covers 28 tasks across five environment types — block, chamber, switch, hanoi, and bowling — implemented in MuJoCo with both CPU and GPU-accelerated MJWarp versions. The GPU version is what makes the study practical: on a single H100 GPU, OCBench collects synthetic trajectories at up to 200K environment steps per second, fast enough for streaming-data training in which an agent learns from a continuous stream of freshly generated data. That setup lets the authors probe how algorithms and design choices behave under effectively unlimited data, rather than under a fixed set of a few hundred human demonstrations.

Overfitting can help, and smaller datasets can win

The first mystery is the one in the title: BC performance often keeps improving as the model overfits more to its dataset. OCBench reproduces this in controlled settings, and the paper uses the benchmark to test explanations rather than assume one. A natural candidate is that the training or validation action error simply tracks performance, but the reported plots comparing success rate against training and validation flow losses and action MSEs on datasets of different sizes show that the MSE metric does not fully explain the dataset-size-reversal phenomenon — the case where a smaller dataset outperforms a larger one.

This distinction matters for how readers should interpret the result. The paper is not claiming that overfitting is generally desirable; it is showing that in these controlled imitation settings, the usual proxy metrics do not reliably predict task success, so a policy that looks worse by loss can behave better in the environment. The evidence is a comparison of success rate against several loss and error measures across dataset sizes, not a claim that loss is irrelevant in every setting.

Closed-loop control fails where action chunking succeeds

The second mystery concerns control structure. A fully closed-loop policy re-plans from the current observation at every step; a partially open-loop policy with action chunking commits to a short block of actions before re-planning. The paper reports that the latter can substantially outperform the former even when there is no inference delay, and that adding history conditioning to closed-loop policies does not necessarily fix the problem. OCBench reproduces this gap in its controlled tasks, and the pixel-based preliminary results show the same pattern on a visual task.

The practical reading is that the failure is not about latency or missing information. History conditioning gives a closed-loop policy strictly more input, yet the paper reports it can still hurt. That points to how the policy behaves once it departs from the demonstration distribution, which is the thread the paper follows in its later analysis of test-time behavior.

Large networks and feature scaling both change outcomes

The third mystery is that BC on human-collected data often requires a much larger network than BC on synthetic data, even for state-based tasks. The paper examines whether flow matching alone explains this need for large-capacity networks and reports that it does not fully account for it. The benchmark's default models are residual MLPs of size 4096-wide × 8 layers, with 1024-wide × 8 layers for the lighter tasks, trained with action chunks of length 25 (length 3 for the lite tasks).

The fourth mystery is feature sensitivity. The authors compare four input configurations on block-double-task1 and hanoi-single-task1 under streaming data: handcrafted scaling of each state dimension, standardized dimensions, and standardized dimensions with the xyz coordinates scaled by 10 or by 0.1. All four contain the same information and differ only in relative scale, and their training and validation metrics — flow losses and MSEs — are almost indistinguishable. Task performance, however, differs substantially.

The authors' stated hypothesis is that different features lead to different test-time generalization. To support it, they measure the MSE between policy actions and oracle actions at test-time states of hanoi-single-task1, regenerating a fresh oracle plan at each such state. Policies with different input features behave differently at test time despite nearly identical training metrics, and this test-time MSE correlates better with performance than training or validation MSE. The authors note that test-time MSE is still imperfect because it ignores multimodality.

What the evidence covers and what it does not

The main analyses are state-based, chosen for computational reasons, and the visual results are presented as preliminary. The paper's own framing is that the phenomena being reproduced were originally observed with visual data, so the pixel-based results on visual-block-single-task1 and visual-hanoi-single-task1 are a check that the benchmark extends beyond state inputs rather than a full visual study. The benchmark tables for BC, flow BC, and FSQ BC are likewise offered as a reference averaged over two seeds, not as a leaderboard, because the paper's focus is analysis rather than algorithm comparison.

The most important scope limit is stated by the authors themselves: OCBench's scripted policies are designed to have properties similar to human demonstrations, but they are still scripted and do not perfectly mimic human behavior. The benchmark is positioned as a fast, convenient playground that complements real-world research, not a replacement for it. That boundary qualifies every reproduction claim: the phenomena appear in a controlled surrogate for human data, which is what makes them studyable, but the surrogate is not the real thing.

The priority the paper establishes

The paper's most useful takeaway is a shift in what to measure. Across the mysteries, the metrics that look most natural — training loss, validation loss, and even action MSE on held-out data — fail to explain the observed differences in success rate, while behavior at test-time states does a better job. The feature-scaling comparison makes this concrete: four configurations with the same information and nearly identical training and validation metrics produce substantially different task performance, and test-time MSE tracks that difference better.

For readers deciding what to trust, the paper's ordering is therefore: prefer evidence about how a policy behaves on the states it actually encounters, and treat overfitting, dataset size, control structure, and feature scaling as variables that change that behavior rather than as proxies for quality. The benchmark's contribution is making that comparison runnable at up to 200K environment steps per second under effectively unlimited data, so that the next hypothesis about a behavioral cloning mystery can be tested rather than argued.

Sources used in this answer