Unlearning as a proxy for invention
The paper asks a narrow but demanding question: could a large language model invent a foundational algorithm like Dijkstra's or Euclid's if it had never been exposed to it? Retraining a model without those algorithms would be computationally prohibitive, so the authors use LLM unlearning instead. Unlearning modifies a pretrained model so that it no longer directly recalls a target algorithm, while leaving the rest of its knowledge intact. The unlearned model is then given application tasks with efficiency requirements tailored to the target algorithm and asked to recover it.
This design does not guarantee that the target knowledge is fully removed. The authors acknowledge that unlearning is not equivalent to retraining a model without the target knowledge, and they treat the unlearned model's recovery rate as an approximate upper bound on what the model could invent from remaining knowledge. That framing matters: a low recovery rate is evidence of limited inventive ability, but a high one would not prove true invention.
Three hint levels and a verifier
Recovery is evaluated in a pipeline with two phases. In the unlearning phase, a pretrained model is transformed into an unlearned model that, under the authors' tests, no longer recalls the target algorithm for concept-level queries. In the recovery phase, the model receives an application task at one of three hint levels: no hint, a high-level hint, or a step-by-step hint. The task specifies input format and complexity requirements, such as inventing an O(N^2) single-source shortest path algorithm for non-negative graphs.
Submissions are checked automatically for correctness and runtime, and every accepted submission is also reviewed manually. The authors report that passing a non-exhaustive test suite does not guarantee correctness: one Gemma-4-12B submission for a majority-vote task passed automatic evaluation using a fixed-seed sampling method that would fail in the worst case, and such false positives are not counted as successful recoveries. They also manually inspect submissions to confirm they implement the target algorithm or an equivalent variant, and report finding no distinct algorithm that met all requirements among reviewed submissions.
The authors introduce a generative verifier that explains why a failing submission fails without revealing the correct solution. It is meant to sustain the model's reasoning strength and avoid a phenomenon they call thought collapse, in which continued revision stops producing useful progress.
Nearly half of algorithms stay unrecovered
The central quantitative result is that, although unlearning does not guarantee full knowledge removal, LLMs fail to recover nearly half of the target algorithms. This figure is the recovery rate under the paper's evaluation pipeline, not a measure of how much knowledge was removed. The two are different: unlearning may leave residual knowledge while the model still fails the recovery task, and the authors use the recovery rate as an approximate upper bound rather than as a direct measure of retained knowledge.
The authors also observe that recoverable algorithms tend to be simple in structure or core ideas, whereas the unrecovered ones are less straightforward. This is a descriptive pattern across the algorithms in their collection, not a controlled test of why simplicity matters.
Suppressing the algorithm name alone hurts recovery
A striking comparison comes from a condition in which the algorithm's name is suppressed during decoding without any unlearning. In that setting, the models' recovery rate drops dramatically, by 19–39%. Because model parameters are unchanged in this condition, the drop cannot be attributed to knowledge removal. The authors interpret it as evidence that models overrely on memorized knowledge and familiar retrieval cues: when the name that normally triggers recall is unavailable, performance falls sharply even though the underlying knowledge should still be present.
This result is the paper's clearest signal that successful algorithm solving in these models is closely tied to recall of a named procedure rather than to reasoning from first principles. It also qualifies how the unlearning results should be read: part of the difficulty in recovery may reflect dependence on retrieval cues rather than absence of the relevant concepts.
Verifier feedback sustains reasoning but not coverage
The generative verifier improves recovery rates by keeping the model's reasoning from collapsing during repeated revision. However, the authors report that continued revision does not expand algorithm coverage within their evaluation budget. In other words, the verifier helps the model stay on task and refine submissions, but it does not enable the model to reach algorithms it otherwise fails to recover. The authors read this as a limitation in self-reflection and its ability to broaden algorithmic exploration.
This distinction matters for readers comparing the verifier to other test-time compute strategies: the verifier is a support for sustained reasoning, not a mechanism that closes the gap on unrecovered algorithms.
What the upper bound does and does not cover
The authors' conclusion is deliberately bounded. They treat the unlearned model's recovery rate as an approximate upper bound and conclude that current LLM systems still have limited ability to make foundational algorithm innovation. The experiments cover a limited set of settings and algorithms, and the authors state that it remains unclear whether the findings generalize to other scientific domains or larger models. Unlearning also does not guarantee equivalence to a model retrained without the target knowledge, so the upper bound is approximate rather than exact.
Within those limits, the paper's contribution is a way to separate knowledge-based algorithm solving from algorithmic discovery. The recovery rate after unlearning, and especially the sharp drop when only the algorithm's name is suppressed, points to reliance on memorized procedures as the main obstacle the authors identify.
The name-suppression gap is the key signal
For readers deciding how much to trust claims about LLM-driven algorithm discovery, the most informative comparison in this paper is not the overall recovery rate but the gap between full access and name-suppressed decoding. A 19–39% drop from suppressing a name, with no parameter changes, indicates that much of the model's apparent algorithmic competence depends on retrieval cues. The unlearning results then show that even when direct recall is suppressed, nearly half of the target algorithms remain unrecovered, and the verifier improves sustained reasoning without expanding which algorithms can be recovered. The paper's priority is therefore to distinguish recall-driven success from genuine reinvention, and its evidence places current systems closer to the former.
Sources used in this answer
Can Large Language Models Reinvent Foundational Algorithms?
Using unlearning to suppress direct recall, LLMs fail to recover nearly half of target foundational algorithms, and suppressing the algorithm name alone drops recovery by 19–39%, suggesting overreliance on memorized knowledge.