[NeurIPS 2025] VL-Calibration: Solving LVLM Overconfidence via Decoupled Visual-Reasoning Trust

TL;DR

Large Vision-Language Models (LVLMs) are notoriously overconfident, often hallucinating facts about images with high certainty. VL-Calibration breaks this cycle by forcing models to separate their "visual trust" from their "logical trust." By using an innovative reinforcement learning framework that measures visual grounding through image perturbations, this work reduces calibration error by up to 76% while simultaneously boosting raw reasoning accuracy.

Problem & Motivation: The "Mixing" Problem

When a model gives a wrong answer to a visual question, where did it go wrong?

1. Perception Failure: It didn't "see" the object correctly.
2. Reasoning Failure: It saw the object but failed the logic.

Traditional calibration treats these as one single value. In LVLMs, language priors (the model's tendency to guess based on text patterns) often drown out visual uncertainty. Previous SOTA methods like RLCR or SaySelf optimize a holistic score, which acts as a "black box" for uncertainty.

Methodology: The Power of Decoupling

The authors introduce a two-phase inference structure. Instead of one output, the model generates:

• Visual Phase: A dense caption ($z_{vis}$) and a visual confidence score ($c_{vis}$).
• Reasoning Phase: A chain-of-thought ($z_{reas}$) and a reasoning confidence score ($c_{reas}$).

The Secret Sauce: Intrinsic Visual Certainty

Since we don't have ground-truth labels for "how certain a model should be about a specific image," the authors use two internal signals:

1. Visual Grounding (KL-Divergence): They mask 80% of the image. If the model's output distribution changes significantly, it was actually "looking" at the image. If not, it was just hallucinating based on text.
2. Internal Certainty (Entropy): A measure of how "spread out" the model's token probabilities are.

These are combined into a Visual Certainty Reward and used in a GRPO (Group Relative Policy Optimization) loop. Furthermore, they use Token-level Advantage Reweighting (TAR) to heavily penalize tokens that exhibit high visual uncertainty during errors.

Experiments & Results

The framework was tested across 13 benchmarks, including DynaMath, MathVista, and MMMU-Pro.

• Calibration Leap: On Qwen3-VL-4B, the Expected Calibration Error (ECE) plummeted from 0.421 to 0.098.
• Accuracy Boost: Unlike many calibration methods that hurt performance, VL-Calibration actually improved accuracy by ~3% because it encourages the model to ground its reasoning in actual visual tokens.
• Architecture Agnostic: The method succeeded on Qwen3-VL (4B, 8B, 30B) and InternVL-3.5.

One of the most impressive results is the Confidence Gap in visually unanswerable problems (where the image is removed). VL-Calibration assigned much lower confidence ($0.218$) compared to the base model ($0.698$), proving it "knows what it doesn't see."

Ablation Insights

The study found that using either Entropy or KL-divergence alone leads to "optimization pathology"—either entropy collapse or explosion. The balanced log-ratio of both is essential for stable RL training.

Critical Analysis & Conclusion

Takeaway: VL-Calibration proves that we don't need massive human-annotated datasets to fix hallucinations. By looking at how the model's own internal probabilities shift when the image is perturbed, we can build a self-supervising signal for trust.

Limitations: The computational cost is slightly higher (11% overhead) due to the dual forward passes needed for KL-divergence. Additionally, while it works on 30B models, the behavior on ultra-large models (70B+) is yet to be fully explored.

Future Outlook: This "decoupling" philosophy could be the standard for future multi-agent or multimodal systems, where each module (vision, audio, touch) reports its own "certainty" before a central "reasoner" makes a final decision.