Confident conflicts are the failure mode being targeted
Small language models are often post-trained as students on reasoning traces from stronger teachers. The paper's starting observation is that strict token-level imitation of teacher traces can force the student to copy continuations that lie far outside its own expected distribution. The authors call the worst of these cases confident conflicts: states where the student has low predictive entropy yet assigns low probability to the target token, so it is confident in a different continuation while being pushed to overwrite that preference.
The paper reports that these divergent targets are common when training on teacher-generated trajectories, even when teacher and student come from the same model family, and largely uncommon in student-generated trajectories, which stay closer to the student's own distribution. The conflicts are also not confined to final answers: in mathematical reasoning traces they frequently involve boilerplate tokens such as whitespace, newline characters, LaTeX symbols, digits, and short numerical fragments. This matters because the disruption is concentrated in low-entropy states, where updates can substantially overwrite learnt representations.
Entropy gates how much teacher signal each token receives
Entropy-Aware Mixing replaces the fixed teacher target with a dynamic per-token interpolation of the student and teacher distributions, gated by the student's predictive entropy. The mixed distribution stays close to the student in low-entropy states and shifts toward the teacher as student entropy increases. The paper implements both convex and geometric interpolations.
The authors position this against two alternatives. Reweighting the SFT loss by student entropy reduces the influence of low-entropy states but can amplify the mismatch between teacher-forced training contexts and the student-generated contexts seen at inference. Mixture-based objectives that interpolate student and teacher distributions with a fixed coefficient avoid that mismatch but apply guidance uniformly, limiting how much stronger guidance can be injected. Entropy-aware mixing is presented as eliminating the need to mask confident conflicts or to rely on small uniform interpolation coefficients.
The same mechanism is tested offline and on-policy
The paper implements the mixture in two training regimes. In the offline setting, it constructs a fixed dataset of reasoning traces by sampling from the entropy-aware target, then trains the student with standard supervised fine-tuning. To generate data efficiently it uses speculative decoding, with the student as draft model and the mixed distribution as target, preserving the exact marginal target distribution. Sampling from the convex and geometric mixtures yields two datasets, ConMix-SFT and GeoMix-SFT.
In the online setting, the mixture becomes the per-token target of an on-policy forward-KL objective, optimised using student-generated samples and yielding ConMix-FKL and GeoMix-FKL. In practice the mixture is built over the renormalised top-64 teacher distribution, and a stop-gradient operator treats the mixture as fixed during training because it depends on the student. The paper reports that the teacher's top-64 tokens already capture at least 90% of the probability mass early in training and rapidly approach nearly full mass, making this an efficient approximation to full-distribution mixing.
Two implementation choices bound how precisely the gate is computed. The student's entropy is approximated using only the top-K logits rather than the full vocabulary; the paper uses K=32 and reports that this approximation remains very strongly correlated with exact full-vocabulary entropy across the student-teacher pairs considered. The paper leaves a systematic study of the effect of K and the resulting top-K approximation error to future work.
Math reasoning improves while general capabilities hold up
Across both settings the paper reports that entropy-aware mixing outperforms traditional fixed-teacher supervision on mathematical reasoning and general capabilities. In the offline Qwen3-1.7B experiments on MATH, GeoMix-SFT reaches the best average Pass@64 of 72.73 against 70.39 for teacher-generated traces and 69.96 for student-generated traces, and the best average Avg@64 of 40.59 against 38.95 and 40.23 respectively. On general-capability benchmarks in the same table, GeoMix-SFT records HellaSwag Avg@8 of 55.07 and MMLU Avg@8 of 62.70, both above the teacher-generated baseline of 51.59 and 61.00.
In the online Qwen3-1.7B-Base experiments, GeoMix-FKL reaches the best average Avg@64 of 36.91 and Pass@8 of 56.34, against 33.85 and 53.99 for plain forward-KL. The comparison set includes OPD, EOPD, Skew-KL, and TAID, and the mixture variants are competitive with or above them on the reported averages. The paper also extends the geometric mixture to reverse-KL distillation through OPD, reporting that GeoMix-OPD remains competitive with OPD with an average improvement of +0.08 Avg@64, +0.04 Pass@8, and +1.40 Pass@64.
These are benchmark scores with specific denominators and sampling budgets, not a general claim about all capabilities. The math results are reported as mean and standard deviation across three seeds, and the general-capability numbers come from HellaSwag, IFEval, and MMLU. The paper states that the full benchmark suite corroborates the main findings.
The best entropy schedule depends on the training source
The paper's central qualification is that the optimal entropy schedule is not universal. Offline-generated traces favour concave schedules, which give greater overall teacher influence. On-policy training favours linear or convex schedules, which concentrate teacher guidance in high-entropy states. In the reported hyperparameters this appears as different mixing transforms for the two regimes: x^2 online and x^(1/4) offline.
This split is the practical decision the paper supports. A reader choosing between offline trace generation and on-policy distillation cannot carry over the same interpolation shape; the schedule has to match how the supervision was produced. The paper frames this as a finding from its analysis rather than a tuning detail, noting that the different entropy-based transformations used in offline and online training make principled or learned interpolation schedules an important direction.
Low entropy does not guarantee the student is right
The paper tests a potential failure mode directly: low student entropy does not necessarily imply correctness, so an entropy-based coefficient could reduce teacher influence exactly when the student is confidently wrong. Grouping Qwen3-1.7B generated trajectories by average entropy, the paper reports that responses in the lowest-entropy quartile are substantially more accurate than the rest, 92.82% versus 64.06%, which suggests higher-entropy sequences may need additional inference-time compute to benefit from better exploration.
The paper then filters trajectories in the lowest-entropy quartile that contain at least one incorrect rollout, yielding 542 prompts, and reports Avg@8 and Pass@8 for student-generated, teacher-generated, ConMix-SD, and GeoMix-SD distributions on those prompts. The authors conclude that preserving confident behaviours does not necessarily lead to worse performance, while noting that future work could combine verifiers to determine when teacher supervision should override the student, especially in safety-critical domains.
Where the evidence stops
The evaluation is limited to student-teacher pairs from the same model family. The paper states that extending the analysis to models from different families could help demonstrate the effectiveness of preserving student behaviour, so the reported gains should not be assumed to transfer across families. The paper does extend evaluation to LFM2.5-350M and Granite-3.2-2B-Instruct to test whether the online distillation gains observed with Qwen3 transfer across model families and student scales, but the stated limitation on same-family pairs remains.
The paper also notes that predictive entropy may simply serve as a proxy for influential updates to the student's shared representation rather than their underlying cause, which is why it calls for principled or learned interpolation schedules. The top-K approximations for both student entropy and teacher support are efficiency choices whose error is not systematically studied here.
The most useful takeaway is therefore a conditional one. Entropy-aware mixing is worth considering when the supervision is off-policy and confident conflicts are likely, and the schedule should be chosen to match the training source: concave with greater teacher influence for offline-generated traces, linear or convex with teacher guidance concentrated in high-entropy states for on-policy training. The paper's own comparison of ConMix and GeoMix variants across the two regimes is the concrete threshold a reader should use before adopting either.
Sources used in this answer
Learning What to Imitate: Entropy-Aware Distribution Mixing
Entropy-aware mixing interpolates student and teacher distributions per token using student predictive entropy, stabilising distillation and improving in-distribution and out-of-distribution math reasoning while better preserving general capabilities than fixed-teacher supervision; the optimal entropy schedule depends on the training source, with offline traces favouring concave schedules and on-policy training favouring linear or convex schedules.