The assistance dilemma is a measurement problem
A good tutor withholds the answer long enough for the student to construct it. Give too little help and the student struggles; give too much and the tutor does the work. The paper frames this as the Assistance Dilemma and argues that LLMs resolve it in one direction: pre-training corpora skew toward question-in, answer-out formats, and single-turn RLHF rewards prioritize immediate helpfulness, so withholding information or asking a question whose value appears several turns later is penalized rather than credited.
Prior RL tutors rewarded the student's success on the tutored problem while the tutor's words were still in context. Under that reward, the cheapest way to raise the score is to tell the student the answer, so a tuned leakage penalty judged by another LLM was needed to reduce telling. The reward and the penalty then pull in opposite directions, and how well the tutor teaches depends on their balance. The paper's claim is that the fix lies in the assessment rather than in the penalty weight.
Three conditions make telling stop paying
The paper identifies three conditions: two on how learning is measured, transfer and student-notes-only, and one constraint, cost of telling. It realizes them as a masked near-transfer post-test, in which the student attempts an unseen variant of the tutored problem seeing only its own turns, with the tutor's utterances replaced by "(hidden)". Because the reward can rise only through what the student wrote in its own turns, cognitive offloading is discouraged and the continuous leakage penalty can be replaced by two binary reward gates: factual correctness of the tutor response, and no solution handover.
The learning gain itself is headroom-normalized improvement over the measured prior, referred to in the learning sciences as normalized learning gain, so a gain on a hard problem is worth as much as a gain on an easy one. An efficiency decay scales only positive gains, using separate per-turn budgets for visible tokens (150) and thinking tokens (512), so brevity alone earns nothing and a negative gain passes through undiscounted. The tutor's reasoning tokens are stripped before an utterance reaches the student or the judges, so only externalized behavior is evaluated.
The gates, not the gain reward, reduce handover
A leave-one-out ablation at 4B isolates what each condition does. Removing the binary gates leaves transfer gain nearly unchanged (+0.23 versus +0.25 for full Eduardo-4B) but raises the solution handover rate from 30.6% to 60.7%. Removing masking produces +0.24 transfer gain and a 13.4% leak rate, but the arm is flagged for empty turns and sequence truncation, so its environment metrics understate gain and leak. Removing the near-transfer post-test gives +0.23 transfer gain and a 47.2% leak rate.
The paper's central finding follows: a learning-gain reward alone does not separate teaching from telling, because a behavior that costs nothing is not unlearned. The gates therefore remain necessary, but since nothing in the reward pulls against them, one fixed setting serves all model sizes. The ablation uses one seed, one model size, and one checkpoint per arm, so it identifies which behavior each condition affects rather than establishing that each is necessary.
One recipe trains tutors from 4B to 27B
Eduardo is a multi-turn RL recipe trained against a frozen Llama-3.1-8B-Instruct student and used unchanged from 4B to 27B parameters across two distinct LLM architectures. A single hyperparameter set is used for all sizes, with no per-model tuning. Training uses DPPO for stability over long multi-turn trajectories of up to 22 turns; the paper describes DPPO as a convenience for stable training rather than a contribution.
Each training step processes 16 prompts times 16 rollouts, or 256 trajectories. Over 100 steps the tutor generates roughly 41M tokens, and each trajectory requires 6 judge calls plus 16 student final attempts. A full 100-step run takes approximately 10-12 hours depending on model size. The reward is gameable because it depends on another agent's learning; during development the authors observed behaviors such as answer-echo laundering, where the tutor tells the answer and asks the student to repeat it, and plateaus of no activity from content-free turns collecting unaided gain.
Transfer gain and same-problem gain diverge
The paper reports two different learning measures that readers should not conflate. Near-transfer gain is measured on an unseen variant with the tutor's words masked; same-problem gain is measured on the tutored problem with the tutor's words still in context. Across 494 problems with 10,000 bootstrap resamples, Eduardo-27B reaches +0.29 transfer gain (95% CI 0.25 to 0.32) and +0.46 same-problem gain (0.42 to 0.51), with a 9.3% leak rate. Eduardo-4B reaches +0.25 transfer (0.21 to 0.29) and +0.37 same-problem (0.33 to 0.42) at a 30.6% leak rate.
Base models show the gap differently. Qwen3.8-27B reaches +0.22 transfer and +0.44 same-problem at an 8.9% leak rate, while Qwen3-14B reaches +0.19 transfer and +0.20 same-problem at a 97.8% leak rate. Gemini 3.1 Pro reaches +0.21 transfer and +0.41 same-problem at a 5.5% leak rate. TutorRL-7B, the prior RL baseline, reaches +0.13 transfer and +0.22 same-problem at an 82.8% leak rate. The transfer column is the one the masked post-test was designed to make meaningful; the same-problem column can be raised by telling.
Frontier-level scores at fewer thinking tokens
The post-trained Eduardo-27B matches Gemini-3.1-Pro on MathTutorBench and Claude Opus 4.8 on TutorMoments at 2.4-6.2x fewer thinking tokens than frontier models, which the paper notes matters for interactive tutoring. On the model-size versus MathTutorBench tutoring quality plot, the 27B model approaches top-tier models at 0.79 versus 0.80 for Gemini 3.1 Pro, using one recipe and one hyperparameter set. The paper also reports that the 27B model achieves the highest pedagogical reward-model score and win rate over human teachers on MathDial among all tested tutors and is the strongest TutorMoments tutor under a plain prompt.
Token efficiency is not only a cost story. The truncation diagnostics show why: base Qwen3.5-4B produced empty turns 70% of the time and Qwen3.5-9B 20%, because reasoning overran the per-turn cap of 4,192 tokens including visible output. All Eduardo models stay below 11% empty turns, and Eduardo-27B below 1%. Arms exceeding 15% on empty turns or sequence truncation are flagged, and their environment metrics understate gain and leak.
Justification is learned, support fading is trained out
Without being named in the reward, the model more than doubles its use of the push-for-justification teacher move. In the example transcripts, Eduardo-27B asks the student to articulate the conceptual distinction between an arithmetic and a geometric sequence, while the base model states the full explanation itself. The reward never names prompting for justification; the paper attributes the behavior to the masked transfer objective.
Support fading, such as assigning independent work, is trained out. Its payoff lies beyond a single-problem dialog episode, so a reward computed within one episode provides no incentive for it. The paper also notes an affect gap: optimizing for cognitive gain alone gives no incentive for socio-emotional moves such as rapport or encouragement. These are consequences of the reward design, not incidental failures of the trained models.
Where the evidence stops and what to weigh
The evidence is bounded in several ways that matter for interpreting the numbers. The student is a single frozen Llama-3.1-8B-Instruct model, so training against one student risks overfitting to its specific failure modes; the paper calls for future training against diverse student ensembles. Near transfer is measured within one domain, math. The student's errors are genuine model failures rather than scripted misconceptions, and LLMs lack the working memory, affective states, and developmental misconceptions of human learners. The ablation is one seed, one model size, and one checkpoint per arm.
Given those bounds, the decision-relevant comparison is not which model scores highest overall but which reward design changes behavior. The 61% versus 30.6% handover gap between the no-gates arm and full Eduardo-4B, at nearly identical transfer gain, is the paper's sharpest threshold: if a tutoring reward can be raised by telling, it will be. The masked near-transfer post-test is what makes the transfer column resistant to that shortcut, and the two binary gates are what keep handover low without a tuned penalty. Readers evaluating a tutoring RL setup should check whether the reward is computed with the tutor's words still in context, because that single design choice determines whether the reported learning gain reflects teaching or telling.
Sources used in this answer
The Assistance Dilemma: Learning to Teach via Multi-Turn Reinforcement Learning
A learning-gain reward alone does not separate teaching from telling; a masked near-transfer post-test plus two binary reward gates reduce solution handover while preserving out-of-domain transfer, and the resulting Eduardo recipe trains 4B to 27B tutors that match frontier models at 2.4-6.2x fewer thinking tokens.