AdaTTA: Revolutionizing Sequential Recommendation with Adaptive Test-Time Augmentation

TL;DR

Sequential recommendation models often struggle with data sparsity. While Test-Time Augmentation (TTA) helps, current methods use a "one-size-fits-all" approach. AdaTTA introduces a Reinforcement Learning agent that looks at a user's specific history and picks the perfect "makeup" (augmentation operator) for that sequence. The result? Up to 26.31% better performance than the best existing TTA methods without ever retraining the underlying model.

The "One-Size-Fits-All" Fallacy

In the world of Sequential Recommendation (SR), we try to predict a user's next move based on their past. But what if the past is too short? Or too noisy? Standard TTA tries to fix this by creating "noisy" versions of the input at inference time and averaging the results.

However, the authors' empirical study reveals a critical flaw:

• Short sequences benefit most from TMask-R (removing items).
• Long sequences perform best with Insert (adding items).
• Specific behavioral clusters might even perform best with no augmentation at all.

Existing methods ignore this diversity, applying the same transformation to every user.

Methodology: The AdaTTA Framework

AdaTTA frames augmentation selection as a Markov Decision Process (MDP). Instead of a human choosing the operator, an RL agent does the heavy lifting.

1. Multi-View State Representation

To make a smart choice, the agent needs to "see" the sequence. AdaTTA uses a hybrid state:

• Semantic View: High-level intent captured via mean-pooling embeddings from the backbone (e.g., SASRec).
• Statistical View: Raw structural data like sequence length, item diversity, and noise levels.

2. Actor-Critic Policy

The agent uses an Actor-Critic architecture. The Actor picks one of 8 actions (Crop, Reorder, Mask, Substitute, Insert, TNoise, TMask-B, TMask-R), while the Critic evaluates how good that choice was to refine future decisions.

3. The Joint Macro-Rank Reward

RL in recommendation is notorious for sparse rewards. AdaTTA solves this with a two-pronged reward:

• Macro Reward: Points for improving top-K metrics (Hit@10).
• Rank Reward: Points for moving the correct item higher in the list, even if it hasn't broken into the top-K yet. This provides a "dense" signal that guides the agent more effectively.

Experimental Results: A New SOTA

The researchers tested AdaTTA against two powerful backbones: SASRec (Transformer-based) and GRU4Rec (RNN-based).

Efficiency vs. Effectiveness

One might worry that running an RL agent at inference time is too slow. AdaTTA proves otherwise. While it is ~1.48x slower than the fastest TTA method, the absolute time is negligible (e.g., 0.93 minutes for the entire Home dataset test set), making it highly viable for real-world production environments where accuracy translates directly to revenue.

Critical Insights & Future Work

Why does it work? AdaTTA acts as an "intelligent filter." It recognizes when a sequence is highly certain (requiring little change) or highly ambiguous (requiring diverse views). The ablation studies confirm that both the statistical features and the rank reward are non-negotiable for success.

Limitations: Currently, the framework selects a single operator per sequence. Future iterations could explore multi-step augmentation (applying a sequence of operators) or generative augmentation where the RL agent fine-tunes a diffusion model's parameters at test time.

Conclusion

AdaTTA proves that Test-Time Augmentation is not just a "free lunch"—it's a lunch that tastes better when it's made-to-order. By moving away from fixed strategies and embracing sequence-level adaptation, AdaTTA sets a new benchmark for inference-time efficiency in recommender systems.