The Amazing Stability of Flow Matching: Why Your Model Might Need 50% Less Data

TL;DR

Is more data always better? In the world of Flow Matching (FM), the answer might be "not necessarily." This paper uncovers a "Surprising Stability" in FM models: whether you prune half the dataset, swap the data for a different one in the same domain, or shrink the model architecture, the generated images remain remarkably consistent. Using a new Cluster-balanced pruning method, the authors even managed to improve image quality (FID) while using 50% less data.

Problem & Motivation: The Quest for Generative Reliability

Training state-of-the-art generative models like DiT (Diffusion Transformers) or Flow-Based models requires staggering amounts of compute. Recent findings in Diffusion models suggested they have a "geometry-aligned basis" that remains stable even on small data subsets.

However, Flow Matching behaves differently. Instead of following the entropic-transport (Schrödinger bridge) logic of Diffusion, FM fits a velocity field $u(x,t)$ used to transport noisy latents to a clean manifold via an ODE. The authors asked a critical question: Is this velocity field robust enough to survive radical changes in training conditions?

Methodology: Stress-Testing the Flow

The researchers tested FM stability through a series of "Perturbations":

1. Data Pruning: Reducing the dataset by 50% using Random, Gradient-norm, Loss-based, or Cluster-based metrics.
2. Disjoint Subsets: Training two separate models on two completely different halves of the data.
3. Architectural Shifts: Comparing a massive 675M parameter DiT-XL to a tiny 33M DiT-Small, and even switching from Transformers to U-Nets.

Informed Pruning Strategies

The authors introduced Cluster-based scoring (Clust). By using CLIP to embed images and k-means to group them, they could choose to:

• Proportional ($Clus{t}_p$): Keep the original data distribution.
• Balanced ($Clus{t}_b$): Equalize the representation of different clusters (e.g., ensuring rare features are represented as much as common ones).

Figure 1: (a) Models trained on disjoint data and (b) different architectures still produce visually similar results for the same seed.

Experiments & Results: Less is More

The results were counter-intuitive. In many deep learning tasks, "high-loss" or "high-gradient" samples are considered the most informative. In Flow Matching, the opposite was often true:

• The Power of Balance: The Clust_b method (balanced clusters) achieved an FID of 22.80, outperforming the model trained on the full dataset (FID 24.24). This suggests that data quality and balance are more important than sheer volume.
• Identity Preservation: Using ArcFace (a face recognition metric), they found that images generated from the same seed across different models had a similarity score of > 0.79. For context, random pairs only score 0.37.

Figure 2: Visual comparison of different pruning strategies. Notice how identities remain consistent even when the training strategy shifts significantly.

Critical Analysis & Conclusion

Why does this work? (The Intuition)

The authors argue that Flow Matching learns a Global Manifold. When you remove a specific cluster of data, the trajectories in that "local area" get rerouted, but the global structure of the vector field remains intact. This "Global Stability" means that the core "rules" of what makes a face look like a face are learned early and robustly.

Limitations

• Domain Specificity: The study focused heavily on CelebA-HQ (faces). While faces have complex variations, they are a relatively constrained manifold. Whether this stability holds for highly diverse datasets like ImageNet or OpenImages remains to be seen.
• Extreme Pruning: While 50% pruning works, there is likely a "phase transition" point where stability collapses.

Future Outlook

This work opens the door for Efficient Flow Matching. We can now envision training pipelines that use "surrogate" small models to prune datasets before the "expensive" training begins, potentially cutting GPU hours by half without sacrificing—and perhaps even improving—the final generative quality.