Can label-free noise assignment preserve multimodal robot actions?

Diffusion policies collapse even with balanced data

Diffusion policies generate actions by iteratively denoising a noise sample, and because different initial noise can in principle yield different trajectories, they were expected to reproduce multi-modal action distributions. The paper finds this expectation often fails. In Push-T, where the T-shaped object starts 180 degrees from its target, clockwise and counterclockwise pushing are both valid; sampled ten times from one fixed observation, a vanilla policy produced ten trajectories in a single modality.

The collapse survived controls that should have prevented it. The authors repeated the experiment across 10 independent training seeds and enforced exact within-batch symmetry by pairing each demonstration with its mirrored counterpart, yet all 10 vanilla policies collapsed to one action mode, with the chosen mode varying by seed. The same pattern appeared across all seven evaluated tasks, including low-dimensional state, RGB, and point-cloud observations in simulation and two real-world humanoid tasks.

This matters beyond behavioral diversity. If deployment changes obstacles, workspace boundaries, object configurations, or execution constraints, the mode the policy kept may become infeasible while a demonstrated alternative remains valid. A collapsed policy can then fail even though it saw a workable strategy during training.

Independent action-noise pairing mixes denoising routes

The paper's explanation centers on how training pairs actions with noise. Vanilla diffusion training gives each action chunk its own independently sampled noise. The authors argue that this unstructured pairing increases mixing and crossing among diffusion paths, so similar noisy inputs can correspond to conflicting action targets from different modalities. The denoiser then learns averaged responses that suppress modality-specific behavior.

The problem is worse in robot planning because action spaces are dense and low-dimensional, which the paper says significantly increases such mixing and crossing. To test the mechanism, the authors vary the noisy input along the direction connecting the two action-modality centroids and measure how much the predicted clean action changes, a quantity they call the modality-response range. A large range means the denoiser distinguishes inputs tied to different modalities; a small range means those differences are suppressed.

At the noise level where the two modalities begin to separate, an analytical denoiser reached a response range of 4.29, while the trained vanilla denoiser reached only 0.24, or 5.6% of the analytical response. That flattened response is the paper's evidence that the vanilla denoiser is largely insensitive to the noise differences needed to route samples toward distinct modalities.

Hungarian matching couples actions to distinct noise regions

Immiscible Diffusion Policy changes only the pairing step. For each mini-batch, it samples a noise pool the same size as the batch of action chunks, vectorizes and standardizes the action chunks using mini-batch statistics, and uses Hungarian matching to assign each action chunk to the noise sample that minimizes the batch-level squared distance. The matched noise then enters the standard forward diffusion process and the usual noise-prediction loss.

Because the matching only permutes noise samples already drawn for the batch, the batch-level noise marginal is unchanged; what changes is the explicit coupling between actions and noise. The method needs no additional demonstrations and no additional gradient updates, and the policy architecture, loss form, noise schedule, and test-time sampling procedure all stay the same. It is therefore a training-time add-on rather than a new policy design.

The assignment uses no modality labels, but it exploits the clustered structure of robot actions: chunks from the same modality tend to be closer to one another than chunks from different modalities. Minimizing action-noise distance therefore correlates nearby action chunks with coherent regions of noise space, so initial noise can carry information about the corresponding modality. On Push-T, the paper visualizes vanilla noise samples from the two modalities overlapping substantially, while the assigned noise shows clearer modality-dependent organization.

The denoiser regains sensitivity to modality direction

The same modality-response measurement that exposed the vanilla failure also tracks the fix. At the modality-separation noise level, Immiscible Diffusion Policy reached a response range of 1.93, compared with 0.24 for vanilla. That is approximately 45% of the analytical denoiser's response, against 5.6% for vanilla. Near the midpoint between the two modalities, its response was about 8.3 times stronger than vanilla's.

The authors report these improvements were consistent across all 10 training seeds. Read together with the noise-space visualization, they support the paper's proposed mechanism: independent pairing mixes the denoising routes of different modalities and makes the learned policy insensitive to initial noise, whereas assignment organizes the noise space and keeps distinct noise-to-action routes. This is the authors' mechanistic interpretation, supported by the response-range measurements rather than proven by them alone.

Suppressed modes return in rollout distributions

The headline results are measured as the share of evaluated rollouts falling into each action modality. In the three simulated two-modality tasks, vanilla policies concentrated almost entirely on one mode: the dominant-modality proportion reached 97.60% in Push-T, 96.67% in Stack Blocks, and 95.56% in Place Bread, leaving 2.40%, 3.33%, and 4.44% for the non-dominant modality. The dominant modality is identified separately for each training seed.

Immiscible Diffusion Policy raised the non-dominant share from 2.40% to 35.00% in Push-T, from 3.33% to 20.00% in Stack Blocks, and from 4.44% to 33.33% in Place Bread. Across these three tasks, the average non-dominant proportion rose by 26.05 percentage points, an 8.68x increase over vanilla; the abstract summarizes the range as 6.0x–14.6x. On both real-world humanoid tasks, the distribution moved from a single hand modality under vanilla to a balanced 1:1 split between left and right hands.

The four-modality tasks show the same direction. In Place Shoe, the distribution changed from 37.8%, 37.8%, 24.4%, and 0.0% to 24.9%, 29.7%, 29.4%, and 16.0%. In Rotate QR Code, it changed from 57.8%, 35.6%, 6.7%, and 0.0% to 44.4%, 17.8%, 26.7%, and 11.1%. In both tasks the method recovered the modality that was entirely absent from vanilla rollouts.

Task success holds, with a cost on recovered modes

On the two-modality simulated tasks, task performance was approximately unchanged. Push-T coverage moved only from 95.16% to 95.39%, Stack Blocks success stayed at 96.67%, and Place Bread success changed from 100.00% to 97.78%. On the real-world tasks, the 1:1 hand balance came with success changing from 75% to 60% on Fruit-to-Plate and from 100% to 90% on Can-to-Bin.

The four-modality tasks show larger drops: 100.00% to 93.33% in Place Shoe and 86.67% to 73.33% in Rotate QR Code. The paper's modality-conditioned analysis attributes this mainly to the newly restored modality rather than to general degradation. Vanilla produced no rollouts from M(4) in either task; the recovered M(4) reached a conditional success rate of 85.71% in Place Shoe but only 40.00% in Rotate QR Code, indicating the recovered behavior was still being learned. On the other three modalities, which both methods generated, success rates were largely comparable.

The authors acknowledge this as a limitation: recovered modalities may suffer low success rates during restoration, and achieving equally high-quality planning across all preserved modalities is left to future work.

When assignment starts shapes the balance

A practical choice is when to switch from vanilla training to assignment-based training. The paper's Push-T ablation compares activation at epoch 0, 50, and 100. Activating at epoch 0 achieved similar modality balance but reduced coverage by 5.13 percentage points under the same training budget, suggesting a short vanilla warm-up helps task acquisition without sacrificing modality preservation. Delaying to epoch 100 kept coverage but increased the dominant-modality proportion by 12.31 percentage points.

The authors operationalize this with a shared criterion across tasks: activate assignment at the first evaluated checkpoint where the task metric reaches at least 60%, provided every demonstrated modality still has nonzero empirical rollout frequency. Because task metrics are normalized to [0,1], this deliberately non-saturated threshold requires substantial progress beyond the initial low-performance regime while leaving training time for assignment. The modality-presence condition ensures assignment begins before modality support is lost, and the shared rule avoids tuning activation epochs per task.

Preservation is the priority, not uniform quality

The paper's most decision-relevant comparison is between coverage of action modes and the quality of each mode. Immiscible Diffusion Policy consistently shifts rollout frequency toward suppressed modalities and recovers modes vanilla never produced, at roughly unchanged performance on the two-modality tasks and with modest overall drops on the four-modality tasks. The conditional results locate the cost: the recovered M(4) mode was the weakest, at 40.00% success in Rotate QR Code.

For a reader deciding whether this add-on fits a deployment, the relevant split is whether the risk lies in losing an alternative mode or in executing every mode well. Where an infeasible preferred mode at deployment would leave a collapsed policy with no fallback, the paper's evidence favors assignment, since it restores modes without changing architecture or inference. Where every preserved mode must meet a high success bar, the low conditional success of newly recovered modes is the binding constraint, and the paper leaves closing that gap to future work.

Sources used in this answer