[Technical Report] Fish Audio S2: Redefining Controllability in Open-Source TTS through RL Alignment

Executive Summary

Fish Audio S2 represents a significant leap for open-source Speech AI. It is not just another TTS model; it is an instruction-following engine capable of generating multi-turn, multi-speaker dialogues with Hollywood-level expressiveness. By combining a Dual-Autoregressive (Dual-AR) architecture with GRPO-based Reinforcement Learning, the system breaks the "robotic barrier," achieving a Real-Time Factor (RTF) of 0.195 and a human-likeness score that eclipses closed-source giants like GPT-4o-TTS in specific benchmarks.

Problem & Motivation: The "Uncanny Valley" of TTS

Traditional TTS systems, even those based on large Transformers, often fail in two areas:

1. Fine-grained Control: Using tags like [laugh] or [whisper] usually results in awkward transitions or ignored instructions.
2. Stability: Long-form synthesis (>1 minute) often encounters "hallucinations" (skipping words) or "timbre drift" (the voice morphing mid-sentence).

The Fish Audio team identified that the root cause is the gap between Self-Supervised Learning (SSL) and Human Preference. To solve this, they treated TTS alignment like a LLM reasoning task, using RL to penalize errors and reward naturalness.

Methodology: The Core Innovation

1. Dual-Autoregressive (Dual-AR) Architecture

Handling high-fidelity audio (44.1kHz) requires dense tokenization. If a model predicts all 10 layers of an RVQ (Residual Vector Quantization) codec at once, the sequence length explodes. S2 solves this with a two-track system:

• Slow AR (Temporal Semantic Backbone): A 4B Qwen-based model that predicts the "base" semantic token.
• Fast AR (Depth-wise Acoustic Decoder): A lightweight 4-layer Transformer that predicts the remaining 9 layers of acoustic detail conditioned on the Slow AR's hidden state.

2. The Multi-Reward RL Framework (GRPO)

The most groundbreaking part of S2 is its use of Group Relative Policy Optimization (GRPO). Unlike standard RL that needs a massive "Value Model," GRPO calculates rewards by comparing a group of outputs.

• Semantic Reward: Penalizes word skipping and hallucination using an internal ASR.
• Acoustic Preference Reward: Scores the "cleanness" and recording quality of the audio.
• Speaker Similarity Reward: Ensures the voice stays consistent with the reference clip.

Experiments & Results: Setting New Benchmarks

In the Audio Turing Test (ATT), which measures if humans can distinguish AI from real speech, Fish Audio S2 achieved a mean score of 0.515. In comparison, GPT-4o scored significantly lower in this specific metric of naturalness.

Key Objectives:

• Long-Audio Stability: On the Long-TTS-Eval benchmark, S2 maintained the lowest Word Error Rate (WER) across news, academic papers, and literature.
• Instruction Following: In the "Fish Audio Instruction Benchmark," S2 showed a 93.3% Tag Activation Rate, meaning it almost never misses a [laugh] or [inhale] instruction.

Production Readiness: SGLang Integration

The model is optimized for SGLang, supporting:

• RadixAttention: Caching reference voices so that repeated generations with the same "voice clone" are instantaneous.
• Streaming: Achieving TTFA (Time-to-First-Audio) of <100ms, essential for interactive AI characters.

Critical Analysis & Conclusion

Takeaway

Fish Audio S2 proves that the next frontier of TTS isn't just "more data," but better alignment. By repurposing ASR and Quality models as RL reward functions, the team has created a feedback loop that trains the model to understand the emotional nuance of text.

Limitations & Future Work

While S2 dominates in Chinese and English, performance in "low-resource" languages (under 1,000 hours of data) still lags behind giants like ElevenLabs. Future iterations will likely focus on scaling the data pipeline for a broader range of global dialects.

Fish Audio S2 is now open-source on Hugging Face and GitHub, providing a powerful foundation for developers building the next generation of digital humans.