How should affordance heads be wired into vision-language-action policies?

A stop-gradient affordance head with an actively initialized residual bridge lifts LIBERO success to 96.2%, beating the headless base.

Direct answer

Wire the affordance head so it reads the backbone through a stop-gradient and re-injects an intermediate head feature into the action expert through an actively initialized residual bridge. In this study that wiring reached 96.2% average LIBERO success, versus 93.1% for the headless base, while letting affordance gradients reach the backbone dropped the policy to 85.5%.

1sources cited

This article was generated with WisPaper-powered search and paper analysis.

Stop-gradient is the precondition, not a detail

The paper's first finding is protective rather than additive. The affordance head reads the backbone's hidden states through a stop-gradient, so the affordance loss updates only the head's parameters and never the backbone. The authors describe this as asymmetric protection: the backbone's language grounding is the scarcest resource in the system, bought with internet-scale pretraining, while the head is cheap and can adapt to the backbone instead.

The cost of skipping this step is measured. Letting affordance gradients reach the backbone drops the policy below the headless base, 85.5% versus 93.1% average LIBERO success. In the authors' preliminary experiments the collapse was sharper still, with goal-conditioned success falling from 94% to below 32%. The stop-gradient is therefore a precondition for any benefit from the head, not one design choice among several.

Initialization outweighs injection topology

With the backbone protected, the study runs a same-budget 2x2 ablation crossing injection topology (concatenation versus residual) with bridge initialization (zero versus random). All four runs share data, budget, and seed; only the head wiring differs. The paper reports that initialization is the dominant lever.

The four wirings and their average LIBERO success rates are CAT-LAZY 93.8%, RES-LAZY 93.6%, CAT-ACTIVE 95.8%, and RES-ACTIVE 96.2%, against the headless base at 93.1%. Both actively initialized wirings beat the base, CAT-ACTIVE by 2.7 points and RES-ACTIVE by 3.1 points on average, with gains concentrated on instruction disambiguation and long-horizon precision. The residual topology adds its own margin on top, chiefly on the Long suite, because its identity path hands the pretrained visual features to the expert untouched while concatenation re-mixes and slightly degrades them.

A zero-initialized bridge stays runtime-inert

The zero-initialized bridge is the conservative choice in adapter design, since the policy starts exactly at the base model. The paper argues this orthodoxy can silently fail in the auxiliary-head setting. The bridge must earn its growth from the gradient of the action loss, and that gradient is starved by construction: the head offers information the action expert can, with effort, extract from the visual tokens anyway. The marginal value of the bridge at zero initialization is therefore near zero, and the optimizer never leaves the attractor.

Empirically, lazily initialized bridges converge to a few percent of the magnitude of the identity path. An inference-time zeroing probe confirms they are runtime-inert: zeroing them changes nothing, so a lazy bridge acts only as a training-time regularizer. Active initialization removes the chicken-and-egg problem by fiat. The feature is present at full magnitude from step one, the expert's representation forms around it, and the pathway becomes genuinely load-bearing, at the cost of a violent start in which the action loss opens an order of magnitude higher before recovering within a few hundred steps.

The head adds format, not new information

The paper is explicit about the information-theoretic ceiling here. The affordance target is a deterministic function of the observation and instruction, and the head reads the same backbone features the action expert already sees. The injected feature can therefore offer the policy no new information, only a more explicit format. Any benefit must come from the optimization dynamics induced by the head's presence, or from the explicit format easing the action expert's job at run time.

The two initializations select between exactly these mechanisms. A lazy bridge leaves only the regularization effect; an active bridge makes the pathway load-bearing. A second probe supports the format reading: ground-truth affordances fed as an input hurt, which is consistent with the value lying in how the head reshapes the expert's representation rather than in supplying a better map.

Labels come free from simulator contact state

The affordance target answers one question per frame: where will the gripper next make task-relevant contact? The authors extract it from existing LIBERO demonstrations with no human labeling by replaying each demonstration step by step in MuJoCo through robosuite. At every timestep the engine's contact solver exposes the exact geometry pairs in contact; scanning forward from frame t, the first object the gripper contacts defines that frame's interaction target. Its 3D contact location is projected through each camera's calibrated intrinsics and extrinsics to a pixel coordinate, and the label is an isotropic Gaussian of fixed scale centered there, one map per camera view.

Two properties matter for reading the results. The label is a look-ahead signal: within a task stage it is constant, and it switches discretely at stage boundaries, so the head is supervised to predict the immediate motor future rather than the present. And because the Gaussian template is fixed, per-frame label mass is constant, so the prediction task never degenerates into predicting label intensity; only location carries information.

The head reads several depths at once

The head taps the backbone's hidden states at a small set of uniformly spaced layers, on the reasoning that affordance prediction needs both ends of the network's semantic spectrum simultaneously. Selecting the correct object among distractors is a late-layer, instruction-conditioned decision, but placing the peak on the mug's handle to pixel accuracy is an early-layer, appearance-driven one. A single tap would force a choice between the two.

Because a vision transformer is columnar, every layer represents the image at the same coarse token resolution, so dense prediction faces the classic tension between recognition and localization. The head resolves it with a top-down pyramid with lateral connections: each tapped layer is reassembled onto its own grid, earlier layers onto finer grids and later layers onto coarser ones, then fused top-down and decoded to a per-view contact heatmap trained with binary cross-entropy. Instruction conditioning arrives through a second route, since image tokens precede the instruction under causal attention; the pooled instruction embedding is applied as a FiLM modulation to the readouts and to every fusion stage. The injected feature therefore carries an instruction-resolved spatial decision rather than a purely visual saliency map.

One head and one bridge match a three-expert design

The best wiring, RES-ACTIVE, reaches 96.2% average LIBERO success with under 1% extra parameters. The paper compares this to AffordanceVLA, the strongest published system it cites, which introduces a purpose-built three-expert mixture-of-transformers with a dedicated affordance-generation expert and a two-stage curriculum, and which the paper reports at 95.8%. The authors' own reading is that the architecture is not the essential ingredient; the wiring is.

That comparison is the paper's sharpest claim, and it is worth stating precisely. The claim is not that affordance supervision is unnecessary, nor that elaborate architectures are worthless. It is that on this benchmark, with this backbone and this data budget, the wiring bits most papers leave unstated, bridge initialization and injection topology, swing the result by more than the presence of the head itself.

Where these conclusions hold and where they stop

The study is limited to one simulated benchmark, one backbone scale, and one seed per cell, partially mitigated by the two-way replication of the initialization effect and a point-identical retest of the Long suite. Affordance labels also come from privileged simulator state, which is what makes them free here and is not available when labels must be produced some other way. The base policy is a 2B-parameter Qwen3-VL backbone with a layerwise flow-matching action expert, and the reported failure modes it targets are instruction disambiguation and long-horizon placement error.

Within that scope the priority ordering is clear. Protect the backbone with a stop-gradient first, because without it the head is worse than no head at all. Then spend the design budget on initialization rather than topology, since active initialization is the main lever on both topologies and the residual form adds a smaller, mostly long-horizon margin on top. The practical threshold the paper offers is that a zero-initialized bridge converging to a few percent of the identity path is a signal the pathway never became load-bearing, and the inference-time zeroing probe is the check that distinguishes a regularizer from a load-bearing route.

Sources used in this answer

1

Wiring Matters: Injection Topology and Initialization of Affordance Heads in Vision-Language-Action Policies

A stop-gradient affordance head re-injected through an actively initialized residual bridge reaches 96.2% average LIBERO success versus 93.1% for the headless base, while letting affordance gradients reach the backbone drops the policy to 85.5%.