MOMENTA: Cracking the Code of Evolving Multimodal Misinformation
MOMENTA: Mixture-of-Experts Over Multimodal Embeddings with Neural Temporal Aggregation for Misinformation Detection
MOMENTA is a unified multimodal misinformation detection framework that integrates modality-specific Mixture-of-Experts (MoE) and Neural Temporal Aggregation. It achieves state-of-the-art results across four major benchmarks (Fakeddit, MMCoVaR, Weibo, XFacta) by capturing cross-modal inconsistencies and narrative evolution over time.
TL;DR
Static analysis is no longer enough to catch modern fake news. MOMENTA introduces a sophisticated framework that combines Mixture-of-Experts (MoE) with Neural Temporal Aggregation to detect misinformation not just by what it says, but by how it contradicts itself and evolves over time. By modeling "narrative drift" and "momentum," it sets new SOTA records on Fakeddit, Weibo, and XFacta.
Context: Why Static Detection Fails
Most current SOTA models treat a social media post as an isolated island (the i.i.d. assumption). However, misinformation is rarely static; it's a process. A narrative that begins as satire might be "purposed" into a fabrication as it spreads. Furthermore, a perfectly "normal" image paired with a misleading headline creates a semantic gap that simple feature concatenation often misses.
Methodology: Specialized Experts & Narrative Physics
The MOMENTA framework is built on three pillars:
- Expert Specialization (MoE): Instead of a one-size-fits-all encoder, it uses modality-specific MoE layers. This allows the model to route "satire" to one expert and "deepfakes" to another, capturing the extreme heterogeneity of fake news.
- Discrepancy-Aware Fusion: Beyond simple alignment, the model features a dedicated "Discrepancy Branch" that explicitly calculates the element-wise absolute difference and product between text and image embeddings. It asks: Where exactly do these two modalities disagree?
- Narrative Physics (Drift & Momentum): This is the most innovative part. Using overlapping time windows, the model calculates:
- Drift (): The change in aggregated representation between consecutive windows.
- Momentum (): The smoothed magnitude of that drift, reflecting the intensity of the narrative shift.
Experimental Showdown: Stronger Generalization
The researchers tested MOMENTA against four major benchmarks. The results confirm that the "temporal-aware" approach is particularly effective in harder, real-world datasets like XFacta and MMCoVaR.
| Dataset | Accuracy | F1-Score | AUC |
|---|---|---|---|
| Fakeddit | 0.965 | 0.959 | 0.982 |
| 0.956 | 0.956 | 0.981 | |
| MMCoVaR | 0.942 | 0.930 | 0.958 |
| XFacta | 0.905 | 0.905 | 0.928 |
MOMENTA achieved a +1.7% Accuracy boost on XFacta compared to GPT-4o-based baselines, proving that specialized architectures still hold an edge over general-purpose LLMs in specific adversarial tasks.
Critical Insights: Domain Invariance
One of the silent killers of misinformation models is "Domain Shift"—a model trained on Reddit (Fakeddit) often fails on Weibo. MOMENTA solves this via:
- Domain-Adversarial Training: Using a Gradient Reversal Layer (GRL) to force the encoder to ignore platform-specific features.
- Prototype Memory Bank: Maintaining "global class centroids" for "Real" vs "Fake" content that remain stable even when moving across different datasets.
Conclusion & Future Outlook
MOMENTA proves that misinformation detection is moving toward process-centric modeling. By treating the evolution of a claim as a physical system with "drift and momentum," we can identify coordinated campaigns that static models miss.
Future Directions: The authors acknowledge the increased computational overhead of MoE. The next frontier will be Sparsely Activated MoE and integrating User Network Graphs to see not just how the narrative changes, but who is driving that change.
For more details, the official code is available at: https://github.com/Yegi03/momenta