LoopCTR: Breaking the Parameter-Computation Lock in CTR Scaling

TL;DR

In the industrial world of Click-Through Rate (CTR) prediction, scaling usually means "stacking more layers," which equates to "more latency and memory." LoopCTR flips this script. By recursively reusing the same shared layer (Recursive Latent Reasoning) during training and applying supervision at every step, it creates a model that thinks hard during training but can "snap-judge" during inference. The result? SOTA performance with a 31x reduction in FLOPs and 53x lower latency compared to existing heavyweight Transformers.

The Scaling Paradox in Recommendations

Large Language Models (LLMs) proved that "bigger is better." However, in recommendation systems, we face a Latency-Quality Trade-off. Stacking 24 Transformer layers might give you 0.002 more AUC, but if it takes 500ms to score a candidate, the system is undeployable.

Existing models (like HSTU or OneTrans) couple Capacity (Parameters) and Computation (FLOPs). If you want more "thought," you must add more "weight." LoopCTR proposes a decouple: scale computation through Loops, not additional layers.

Methodology: The "Sandwich" with a Brain

LoopCTR organizes the data flow into a "Sandwich":

1. Entry Block: Projects heterogeneous features (user, item, context) into a unified space.
2. Loop Block (The Brain): A shared layer that processes tokens repeatedly. To prevent this shared layer from being too "weak," the authors added:
   • Hyper-Connected Residuals (HCR): Instead of a simple x + f(x), it uses multi-stream adaptive fusion that changes its behavior at each iteration.
   • Mixture-of-Experts (MoE): Expands the parameter pool (knowledge) without increasing the computation per token.
3. Exit Block: Produces the final CTR score.

Why "Zero-Loop" Inference Works

The "secret sauce" is Process Supervision. During training, the model is forced to output a valid prediction after every single loop. This forces the shared parameters to internalize the "refined thoughts" into the early representations. At serving time, you can skip the Loop Block entirely (0 loops), and the model still performs better than traditional deep models because it was trained to be "efficiently smart."

Experimental Battleground

LoopCTR was tested against a massive suite of baselines (DIN, DCNv2, HSTU, etc.) across four datasets.

Key Findings:

• Superior Accuracy: LoopCTR wins on almost every metric (AUC, GAUC).
• The Efficiency Miracle: On the industrial InHouse dataset, LoopCTR(0/3) — the zero-loop version — reached an AUC of 0.7007 while using only 13.38M FLOPs. Compare this to HSTU, which needed 2150M FLOPs to get a lower AUC (0.6960).

The Untapped Frontier: 0.04 AUC Headroom

The most fascinating part of the paper is the Oracle Analysis. The authors analyzed what would happen if the model could choose how many loops to run for each specific user. They found an untapped 0.02 to 0.04 AUC gap.

Interestingly, models trained with fewer loops actually have a higher theoretical ceiling—meaning they develop more diverse representations across the loops, even if their average performance is lower. This suggests that the next big breakthrough in CTR won't be a larger model, but an Adaptive Model that knows when to "stop thinking" for easy samples and when to "loop" for complex user behaviors.

Critical Insight & Conclusion

LoopCTR proves that in recommendation systems, Weight Sharing is not just a compression trick; it's a powerful Inductive Bias. It forces the model to learn more generalizable features that survive multiple passes, mitigating the classic overfitting seen in sparse CTR data.

Takeaway for Engineers: If your Transformer model is too slow for production, don't just prune it. Try looping it. The "train-deep, infer-shallow" paradigm is a game-changer for high-scale ranking systems.