Sequence position is a proxy for elapsed time
Most imitation-learning policies turn demonstrations into fixed-rate observation-action sequences, so the position of a token in the sequence stands in for how much physical time has passed. That works only while the relationship between position and time stays constant. The paper's motivating case is a robot grasping an object from a conveyor belt: the same image displacement can mean very different object velocities depending on whether the two images were captured 20 ms or 200 ms apart, and the right grasp also depends on when the predicted action will actually run.
Real systems vary timing at every stage: sensors sample at different rates and latencies, inference takes time, and controllers execute at different frequencies. The paper treats this not as an external mismatch to correct but as part of the policy input. It formulates the policy as a function of observations, their capture times, and the desired action execution times, so relative timing encodes observation spacing, measurement age, and the requested execution schedule.
Time-aware RoPE keeps token content separate from time
The central design question is how timing should enter the transformer. One option, which the paper calls AddTime, encodes timestamps as sine and cosine features passed through a shared MLP and adds the resulting embedding directly to each token, mixing what a token represents with when it occurs. Chronos Policy instead leaves token content unchanged and uses physical time only as positional encoding in attention: observation tokens use their capture times and noisy action tokens use their queried execution times as rotary coordinates.
Because rotary attention depends on the difference between coordinates, attention then depends on elapsed time between tokens rather than on sequence index. The implementation rotates 16 query-key coordinate pairs using logarithmically spaced periods between 0.1 and 2 seconds, with timestamps measured in seconds relative to the policy request time. The authors argue this lets attention represent observation-to-observation, observation-to-action, and action-to-action timing directly.
Randomized timing broadens the training schedules
During training, the paper temporally randomizes which observations are sampled and when the action chunk begins. Tasks sample observation rates of 12.5 to 33.33 Hz and delivery delays of 0 to 50 ms independently across sensor streams from a 100 Hz dataset; for the Pivoting task, force/torque is sampled from 1 to 100 Hz with 0 to 100 ms delay. Half of the action chunks start with zero delay and the rest sample a start delay up to 100 ms. This exposes the policy to a wider range of sensing and execution schedules and allows learning from asynchronous, multi-rate data without forcing measurements onto a fixed grid.
The ablation shows the two components depend on each other. On Conveyor, temporal randomization alone raised Diffusion Policy's average success across the five perturbed conditions from 6.3% to 26.9%. On Pivoting it did the opposite, cutting the perturbed-condition average from 48.4% to 21.6%, because without timestamps the policy cannot tell what physical intervals produced a sampled history. Once physical time is explicit, randomization helped consistently: the perturbed-condition average rose from 48.8% to 63.1% on Conveyor and from 33.3% to 55.3% on Pivoting.
Dynamic tasks gain most when timing shifts
On the dynamic Conveyor task, where the policy must account for object motion and the time between sensing and execution, Chronos Policy improved nominal success from 73.1% to 82.0% over Diffusion Policy. The gap widened as deployment timing changed: with moderate observation variation, Diffusion Policy dropped to 15.6% while Chronos Policy retained 77.6%; under inference delays up to 200 ms, Diffusion Policy fell to 1.8% while Chronos Policy reached 76.4%. Under the combined perturbation, Chronos Policy retained 32.7% success while Diffusion Policy failed entirely.
The multisensory Pivoting task shows a different pattern. Under mild timing changes the two policies performed similarly, and the advantage of explicit time grew as mismatch increased: success improved from 51.3% to 60.2% under inference delays up to 200 ms, and from 19.6% to 38.4% under the combined condition. The authors explain the difference by task structure: Pivoting does not require continuously predicting a moving target's future position, so timestamps matter mainly when the relative times of vision, proprioception, and force/torque measurements vary enough that sequence positions no longer indicate acquisition time.
RoPE beats additive timestamps under perturbation
Comparing the two time representations isolates how physical time enters the transformer, since AddTime and RoPE share the same timestamps, token layout, diffusion conditioning, and temporal randomization. On Conveyor, AddTime achieved slightly higher nominal performance, 85.8% versus 82.0%, but RoPE performed better in every perturbed condition, with the largest differences under wide sensing variation (45.6% versus 24.0%) and combined perturbations (32.7% versus 7.8%). On Pivoting, RoPE had higher mean success in every condition, including 48.0% versus 23.6% under wide sensing and 38.4% versus 16.7% under combined perturbations.
The authors hypothesize this follows from the simpler separation of content and time, and from pairwise elapsed time directly modulating attention rather than being inferred from additive token features. This is an interpretation rather than a measured mechanism, but it is consistent with the ablation: the two variants differ only in how the same physical timestamps are represented.
Quasi-static tasks keep nominal performance
A natural worry is that adding time could hurt tasks where timing is less critical. On six standard quasi-static manipulation tasks, Chronos Policy and Diffusion Policy achieved comparable performance under nominal timing, so explicit time modeling did not sacrifice accuracy there. The advantage appeared when deployment timing differed from training: averaged across the five perturbed conditions, success improved from 59.7% with Diffusion Policy to 73.4% with Chronos Policy, with the largest gains under wide sensing variation and combined perturbations.
The real-world evaluation points in the same direction. On Dynamic Cup, Peg Insertion, and Pivoting, Chronos Policy achieved the highest task score in every evaluated condition across all three tasks, including observation perturbations, inference latency, and their combination. The comparison included Diffusion Policy with Real-Time Chunking, which improved robustness to inference latency in several settings but does not explicitly account for changes in observation timing, so its benefit did not consistently extend to sensing perturbations. The authors read the two approaches as addressing complementary aspects of temporal mismatch.
Where explicit time matters most
The paper's evidence supports a specific priority rather than a blanket upgrade. Explicit time pays off most when the deployment schedule departs from training: the largest gains appear under wide sensing variation and combined observation-plus-inference perturbations, while nominal performance stays comparable. Within the design, time-aware RoPE is the more robust choice over additive timestamp embeddings, and temporal randomization is only useful once timestamps make the sampled schedules interpretable.
The scope is bounded by the tasks and conditions tested: two simulated manipulation tasks, six standard quasi-static tasks, and three real-world tasks, all with a 16-action prediction horizon at 20 Hz. The authors list extending Chronos policies beyond fixed action rates to variable control frequencies and temporal scaling as future work, so the current results do not establish behavior under those regimes. For readers deciding whether timing should be an explicit policy input, the paper's clearest signal is that the benefit concentrates where observation-to-action timing or cross-sensor timing actually varies.
Sources used in this answer
Chronos Policy: Learning Robot Policies on Physical Time
Conditioning a diffusion transformer on observation capture times and desired action execution times via time-aware RoPE improved success on dynamic and multisensory tasks under timing perturbations while preserving nominal performance on quasi-static tasks.