[CVPR 2025] CARE-Edit: Resolving Multi-Condition Conflicts via Condition-Aware Routing of Experts

TL;DR

Unified image editors often struggle when multiple conditions—like text prompts, spatial masks, and reference images—clash, leading to "color bleeding" or loss of subject identity. CARE-Edit solves this by replacing the static "one-size-fits-all" backbone with a Condition-Aware Routing of Experts (MoE) system. By dynamically assigning specific image tokens to specialized experts (Text, Mask, Reference, or Base), it achieves superior fidelity in complex contextual editing with 10x better data efficiency than previous SOTA models.

Background: The "Static Fusion" Bottleneck

Current diffusion editors like ControlNet or OmniControl treat all control signals (text, masks, references) as equal inputs that are simply concatenated or added to the model's features. This static fusion is problematic because:

1. Task Interference: A text prompt might override a mask boundary, causing "semantic leakage."
2. Timestep Negligence: The importance of a mask (spatial layout) is high in early diffusion steps, while style/reference consistency matters more in later steps. Static models can't adjust this balance.

Methodology: Specialization through Routing

CARE-Edit introduces a modular architecture within a frozen FLUX.1 (DiT) backbone. The core innovation lies in its Sparse Top-K Routing mechanism.

1. Heterogeneous Experts

Instead of identical expert blocks, CARE-Edit uses four specialized units:

• Text Expert: Focuses on semantic reasoning and synthesis via cross-attention.
• Mask Expert: Dedicated to spatial precision and boundary refinement.
• Reference Expert: Handles identity and style transfer using FiLM or cross-modal adapters.
• Base Expert: Maintains global coherence and background stability.

2. The Routing Mechanism

A lightweight latent-attention router analyzes each token and the global task (e.g., "removal" vs. "style transfer") to calculate routing probabilities. Only the Top-K (K=3) most relevant experts are activated for a specific token at a specific timestep.

Figure 1: The CARE-Edit framework routing tokens to specialized adapters within the DiT backbone.

3. Mask Repaint & Latent Mixture

To ensure seamless blending, the Mask Repaint module iteratively refines coarse user masks to match object contours. The Latent Mixture then fuses expert outputs using a learned, timestep-dependent gate that anchors the edit to the original image's structural "base."

Experiments & Results

CARE-Edit was tested against heavyweights like OmniGen2 and EMU-Edit.

• Identity Preservation: On the DreamBench++ benchmark, CARE-Edit achieved a DINO-I score of 0.568 in multi-object settings, surpassing OmniGen2 (0.560).
• Data Efficiency: Remarkably, CARE-Edit reached SOTA performance using only 120K triplets, whereas competitors often require millions of samples.
• Task-Expert Synergy: Empirical analysis proves that the router learned to prioritize the "Mask Expert" for object removal and the "Reference Expert" for style transfer automatically.

Figure 2: Qualitative comparison showing CARE-Edit's superior backgroud-foreground integration compared to other unified editors.

Critical Insight: Why it Works

The "magic" of CARE-Edit isn't just the MoE, but the Expert Specialization. By visualizing attention maps, the authors found that the Base Expert acts as a "global anchor," while the Mask Expert learns to focus its attention strictly within the edit region over time. This disentanglement prevents the "identity drift" common in models that try to process everything through a single set of shared weights.

Figure 3: Visualization of different experts evolving unique spatial attention patterns during training.

Conclusion & Future Outlook

CARE-Edit proves that modularity is the cure for interference in multi-modal generative tasks. By treating different conditioning signals as different "competencies" for specialized experts, it clears the path for highly controllable and high-fidelity image manipulation. Future work will likely look at Dynamic Expert Expansion, allowing models to "download" new experts for specific artistic styles or object categories on the fly.

Takeaway: If you want a model to follow a mask and a prompt simultaneously, don't just feed it both—route the compute to those who know each signal best.