How does contrastive vision on-policy distillation fix VLM visual shortcuts?

Distilled VLMs answer without the visual evidence

The paper starts from an anomaly rather than a benchmark gap. After on-policy distillation, a vision-language model could still answer questions accurately when the visual region needed for the answer was masked out or replaced with noise. If the answer survives removal of the evidence, the model is not demonstrably reading that evidence.

The authors treat this as a shortcut created by how distillation is set up. In on-policy distillation, the student generates its own trajectories and is trained to match the teacher's output distribution on those trajectories. When the teacher is given privileged information, such as an answer-centered crop, that crop is a hint. The student sees the full image and the question, but the supervision it receives is shaped by information it cannot observe or differentiate.

The paper's diagnostic comparison is direct: accuracy is measured under black masking and under Gaussian noise applied to answer-relevant regions, and attention is visualized for the crop-conditioned Vision-OPD setting. The reported observation is that correct answers persist despite broken evidence, and the authors read this as failure to genuinely seek out and rely on visual content.

Why matching the teacher leaves visual use unconstrained

The theoretical argument is about what the optimization target actually constrains. The student distribution is conditioned on visual content, the question, and the preceding answer tokens. The positive teacher's distribution is conditioned on a hint whose mapping from the image is neither differentiable nor observable to the student.

Because the student is pushed toward the teacher's token posterior, the gradient raises the logit of the hint-related target token. The paper states that this gradient imposes no constraint on what the student should optimize with respect to the image. Optimization therefore falls back on the full image and the fixed textual context, and the model learns to fit the available conditions rather than to identify the positive region.

This is the paper's explanation, not an independently measured causal decomposition. It matters because it predicts where a fix must act: not by adding more teacher signal, but by making the student's output depend on which visual region the teacher saw.

A negative teacher makes the crop difference informative

CV-OPD keeps the positive, answer-related crop and adds a second teacher branch that sees a negative region sampled from the same image. The negative region's height and width match the positive region, and its overlap with the positive region is constrained by an IoU threshold. The two teacher branches share parameters and condition on the same student prefix, so the only intended difference between them is the privileged crop.

The student is then trained with a Contrastive Divergence Loss. At each token position the method computes the full-vocabulary Jensen-Shannon divergence between the student and each teacher, then applies a margin-based hinge term that pulls the student toward the positive teacher and pushes it away from the negative one. Once the negative divergence exceeds the positive divergence by the margin, the term drops to zero, so the negative branch is not pushed away indefinitely.

The paper derives that the resulting gradient is proportional to the difference between the positive and negative teacher distributions. In the authors' interpretation, this lets the loss amplify the sampling discrepancy between the two visual inputs and thereby help the model localize the positive region.

Gating concentrates contrast on visually relevant tokens

Applying the contrastive term everywhere would dilute it with sampling noise and stylistic variation. The Discrepancy Aware Gate uses the full-vocabulary divergence between the student and the positive teacher as a token-level evidence score, and activates the contrastive term only at positions whose positive divergence exceeds the batch average.

A second gate operates at the sample level. Because the teacher conditions on the student's prefix, an incorrect rollout can leave the teacher correcting from an off-track prefix. The method keeps an example only if at least one of its rollouts produces a correct final answer; if all rollouts are wrong, all supervision for that example is disabled.

The ablation separates the two mechanisms on a six-benchmark average. Removing the contrastive loss lowers the average from 78.22% to 76.81%, and removing the gate lowers it to 76.88%; removing both gives 75.92%. The paper reads the gate's contribution as evidence that selecting informative tokens provides more effective supervision than treating all positions equally.

The negative crop must be unrelated, not merely nearby

The choice of negative region is not incidental. The paper compares an all-black image, a crop neighboring the target region, and a randomly sampled crop. A neighboring crop reaches a 76.40% average, below the 76.81% variant without the contrastive loss at all. The authors explain that regions near the target may still contain answer-relevant information, so pushing the student away from them supplies conflicting supervision.

All-black reaches 76.76% and random crop reaches 77.20%, while the full CV-OPD configuration reaches 78.22%. The gap between random crop and CV-OPD is attributed to sampling consistency: CV-OPD shares the same negative crop across all eight rollouts of each sample, which the authors argue reduces variation in contrastive gradients.

The gate threshold shows a similar pattern. Using each rollout's mean discrepancy gives 77.08%, only slightly above the 76.88% no-gate variant, because short responses make a rollout-level average sensitive to a few token positions. Using the batch-level mean raises the average to 78.22%.

What the six-benchmark average does and does not show

The headline comparison is an average over six fine-grained visual benchmarks: V* Zoom, HR-Bench 4K, HR-Bench 8K, MME-RW English, MME-RW Chinese, and one further benchmark contributing to the six-benchmark average. The full configuration scores 91.62, 62.01, 84.38, 83.38, 76.99, and 70.93 respectively, for 78.22% on average.

The base 4B model scores 71.12% on the same average, and the all-black negative variant scores 76.76%. These are aggregate scores on the listed benchmarks, not a claim about every task or every model scale; the ablation table reports a 4B model. The paper's own framing is that CV-OPD substantially alleviates the shortcut and consistently improves visual reasoning across the tested benchmarks.

The diagnostic evidence and the benchmark gains answer different questions. Masking and noise tests ask whether the model still answers when evidence is removed; benchmark accuracy asks how well it performs on fine-grained visual tasks. A gain on the latter does not by itself prove that the shortcut is gone, which is why the paper pairs the two.

Where the contrastive signal earns its place

The paper's most decision-relevant comparison is not CV-OPD versus the base model but CV-OPD versus its own weaker variants. A negative crop drawn too close to the target performs worse than adding no contrastive loss at all, and a rollout-level gate threshold barely improves on no gate. The method's benefit depends on two conditions that the ablations make explicit: the negative region must be genuinely unrelated to the answer, and the gating threshold must be stable enough to select informative tokens.

That reframes the practical question. If a distillation pipeline already supplies answer-centered crops to the teacher, the first thing to check is whether the student still answers when those regions are masked. If it does, the evidence in this paper points to contrastive supervision with a shared, unrelated negative crop and batch-level gating as the configuration that produced the 78.22% average, rather than to a nearby crop or a per-rollout threshold.

Sources used in this answer