[arXiv 2026] UniCom: Unifying Multimodal AI via Compressed Continuous Semantic Representations
UniCom: Unified Multimodal Modeling via Compressed Continuous Semantic Representations
UniCom is a unified multimodal framework that integrates visual understanding and generation by utilizing compressed continuous semantic representations. By employing an attention-based semantic compressor to reduce channel dimensions, UniCom achieves state-of-the-art performance in image generation and editing, reaching rFID scores as low as 0.38 on ImageNet.
TL;DR
UniCom represents a significant shift in multimodal modeling by moving away from discrete visual tokens and traditional VAEs. By compressing continuous SigLIP2 features along the channel dimension via a lightweight attention-based module, UniCom creates a unified latent space that is both semantically rich for understanding and mathematically "smooth" for high-fidelity image generation and editing.
The Problem: The Discretization Dilemma
In the quest for a "unified token" that works for both LLMs (understanding) and Diffusion Models (generation), researchers have faced a choice between two evils:
- Discrete Tokenization (VQ-VAE/VQGAN): Easy for LLMs to predict but causes irreversible information loss, leading to blurry textures and poor text rendering.
- Raw Continuous Features (CLIP/SigLIP): Excellent for understanding but extremely difficult to model generatively due to high dimensionality (e.g., 1152-d) and a complex, non-smooth manifold.
UniCom solves this by asking: Can we compress these high-dimensional manifolds without losing the "perceptual" essence?
Methodology: Channel Compression & Transfusion
The core innovation is the Semantic Compressor. Unlike prior works that reduce the number of tokens (spatial downsampling), UniCom finds that reducing the channel dimension (from 1152 to 64) preserves significantly more detail.
1. The Attention-Based Compressor
Rather than a simple MLP, UniCom uses a shallow Transformer-based projector. This allows the compression to be "context-aware"—each patch is compressed with knowledge of its neighbors, preserving structural semantics better than independent patch processing.
Figure 1: The UniCom framework, showing the Semantic Compressor bridging SigLIP features and the Diffusion Decoder.
2. Transfusion vs. MetaQuery
The authors compared two pathways:
- Pathway I (Transfusion): A unified Transformer trained on mixed-modality sequences using Flow Matching.
- Pathway II (Query-based): Using an MLLM to output "MetaQueries" for a separate decoder.
The results were clear: Transfusion (Pathway I) maintains better spatial correspondence and faster convergence, making it the backbone of UniCom.
Experimental Breakthroughs
The model was built using Qwen-2.5-7B and FLUX.1-dev, essentially merging one of the best LLMs with one of the most powerful generative backbones.
Reconstruction & Generation Fidelity
Even with an 18x compression ratio (d=64), UniCom recovers high-frequency details (like fine text) that other semantic-based unified models lose. In text-to-image benchmarks like GenEval and Wise, UniCom rivals or beats specialized generation-only models.
Figure 2: UniCom (d64) maintains facial identity and text clarity far better than older semantic baselines.
Superior Image Editing
One of the most impressive feats is UniCom's performance on KRIS-Bench and WorldEdit. Because it operates in a continuous semantic space, it can handle "intelligent" edits—such as changing an animal's species or aging a person—by reasoning within the latent manifold rather than just performing pixel-level manipulation.
Critical Insight: Why Channel Compression?
The ablation studies (Figure 4 in the paper) prove a vital point for the industry: Spatial tokens are sacred. Reducing the number of tokens from 1024 to 256 causes massive blurring, but reducing the dimensionality of those 1024 tokens from 1152 to 64 has almost no visible impact on reconstruction. This suggests that Vision Transformers currently store a lot of redundant information across their feature channels.
Conclusion & Future Outlook
UniCom proves that we are entering an era of "VAE-less" unified modeling. By treating vision as a compressed continuous dialect of language, we can finally bridge the gap between "seeing" and "creating."
Key Limitation: While efficient, training these models still requires significant compute, and the joint optimization of the compressor/decoder must be handled carefully to avoid manifold collapse.
Takeaway for Researchers
If you are building a unified model, stop quantizing. Look into Channel-wise Continuous Compression. It's the "Goldilocks zone" between LLM efficiency and Diffusion fidelity.