Can neuronal ablation in language models reproduce patterns of aphasia?

Silencing units and watching what breaks

The paper asks whether the pattern of language decline — which abilities survive, which erode gradually, which collapse abruptly — can be predicted from damage to a computational architecture. To test this, the authors remove computational units from language models and measure what happens to next-word prediction and to a 100-prompt fill-in-the-blank (Cloze) battery that separates grammar from word meaning.

Two model families are used. The recurrent models are a 3-layer Small LSTM (about 10M parameters, 580,320 non-embedding parameters) and a larger AWD-LSTM (about 44M parameters, 20.3M non-embedding), both trained on WikiText-103. The Transformers are Qwen 0.5B and Qwen 1.8B, 24-layer dense models evaluated on TinyStories passages. Because these models differ in size, corpus, and tokenizer, the authors report relative accuracy (Acc(k)/Acc(0)) and compare error distributions against an accuracy-matched random-failure baseline rather than relying on raw scores alone.

Ablation is binary and uniform: a fraction k of units is zeroed across all layers, and gradient hooks keep those entries at zero. For LSTMs, whole-unit removal zeroes both the input-to-hidden and recurrent weights plus biases, verified by forward hooks showing max activity of 0.0; at k = 0.5 this removes 70.4% of non-embedding parameters in the Small LSTM and 72.7% in the AWD-LSTM. For Transformers, MLP neuron removal zeroes up/gate projection rows and down projection columns (33.7% of non-embedding parameters at k = 0.5), while attention-head removal zeroes Q, K, V rows and O columns (16.3% at k = 0.5).

LSTMs lose grammar and meaning together

In both LSTMs, whole-unit removal produces a smooth decline with no cliff. The Small LSTM's accuracy falls from 0.2747 at k = 0 to 0.1260 at k = 0.5 and 0.0712 at k = 0.9; the AWD-LSTM falls from 0.3806 to 0.1307 at k = 0.5 and 0.0678 at k = 0.9. On the Cloze battery, grammar and word meaning erode steadily together from k = 0.1, reaching the zero floor on content words by k = 0.4–0.6, after which the models fall back on common function words.

The authors test whether this shape is an artifact of the fine-tuning protocol used as a stand-in for compensatory plasticity. They compare no fine-tuning, brief local SGD, and 500-step AdamW retraining. The LSTM curves stay smooth even under in-domain AdamW retraining, and with zero fine-tuning the Transformer MLP pathway still shows its sharp late collapse without an early retention advantage over the LSTMs. The authors therefore attribute the curve shape to architectural wiring rather than short-term weight recovery.

Connection-level controls explain the LSTM profile. Cutting only the recurrent links from earlier words causes a slow co-decline of grammar and meaning across the full range, whereas cutting the input links that feed the current word up the three-layer chain causes a rapid drop onto function-word defaults once enough serial capacity is lost. Making the LSTM larger raises undamaged accuracy from 27.5% to 38.1% but does not change the failure pattern.

Transformers split into two failure modes

The Transformer results depend on which sub-block is damaged. Under MLP neuron ablation, accuracy is largely retained and then collapses at a late threshold, around k* ≈ 0.3–0.4. Under attention-head ablation, the decline is smoother and sigmoid-like, with an early plateau: Qwen 1.8B attention stays near baseline through k = 0.4 before steepening, and Qwen 0.5B attention behaves similarly.

The authors connect this split to proposed roles inside a Transformer block. Prior interpretability work suggests feedforward MLP layers act as distributed memory stores for word associations, while attention heads gather context from earlier words along a shared residual stream. Under that hypothesis, damaging MLP neurons leaves context intact but impairs retrieval of the exact target word, while damaging attention heads leaves the word store intact but weakens sentence context, pushing the model toward safer, more general words.

The error patterns follow this account qualitatively. MLP-ablated models keep making within-category semantic substitutions — semantic paraphasias — at the undamaged rate before collapsing, alongside rising non-responses; the authors compare this to Wernicke-type aphasia, while noting the parallel is partial because blank and function-word outputs also rise, so fluency is not preserved. Attention-head ablation produces a trend toward loss of semantic specificity, which the authors compare to the semantic variant of primary progressive aphasia (svPPA).

Why error mix matters more than raw accuracy

The paper's central measurement claim is that accuracy alone hides the damage. Even at mild ablation (k = 0.1–0.2), the models' new mistakes already shift toward less common content words rather than ultra-common function words relative to an accuracy-matched random-failure baseline (∆R4 < 0, ∆D1 > 0). This shift is significant from k = 0.1 in both LSTMs and Qwen MLP, and from k = 0.2 in Qwen 1.8B attention.

The authors also compute Fourier spectral metrics β and G to describe how smoothly prediction errors vary across word-frequency ranks. These raw metrics fluctuate non-monotonically and vary more across trials than R4, so the authors base their main cross-pathway comparisons on accuracy-matched, null-corrected ∆R4 and Jensen–Shannon divergence instead. That choice matters for readers: the spectral numbers describe the shape of the error distribution, but they are not the evidence used to separate pathways.

The clinical parallels are assigned from the shared 100-prompt Cloze battery, not from the spectral analysis. The authors describe these labels as descriptive names for error patterns, not claims that the models have comprehension or disease.

An early plateau that resembles clinical reserve

The most specific clinical analogy concerns timing. Early accuracy retention under Transformer attention-head ablation qualitatively parallels the pre-symptomatic reserve before the 40–60% neuronal loss at which clinical symptoms emerge in Alzheimer's disease, as reported by Gomez-Isla et al. (1996). The steadier LSTM decline parallels the symptomatic progression described by DeKosky and Scheff (1990) and Terry et al. (1991).

The authors are careful about the level of this analogy. Alzheimer's disease initially presents with a prolonged pre-symptomatic phase followed by broad, multi-domain decline; the LSTM whole-unit comparison is offered as capturing that second, syndromic dimension — multiple linguistic functions eroding gradually and in parallel without a selective syntax–semantics split — in contrast to the focal pathway dissociations seen under Transformer sub-block ablation.

This is where the paper's practical suggestion sits. Because disease-modifying Alzheimer's therapies require intervention before tau-driven cortical atrophy becomes widespread, the authors propose that comparing patients' word-frequency error patterns on naming or Cloze tests against an accuracy-matched random baseline could help distinguish early structural decline from benign attentional lapses before overall test scores fall. That proposal is a hypothesis derived from model behavior, not a validated clinical tool.

Where the aphasia analogy stops

The ablation is one idealized mode of damage: uniform binary zeroing across all layers. Real disease and stroke can weaken synapses gradually, add noisy signalling before cell death, concentrate damage in specific layers or regions, break connections between regions, and recover through mechanisms unlike standard fine-tuning. The authors state that their clinical parallels describe output-level behavioural analogies rather than one-to-one mappings onto neurocognitive components.

The empirical scope is also bounded. The LSTMs (about 10–44M parameters, trained on WikiText) and Transformers (0.6–1.8B parameters, evaluated on TinyStories) differ in size, corpus, and tokenizer. The 100-prompt Cloze battery separates the overall degradation curves of the MLP and attention pathways, but larger batteries would be needed to test fine-grained differences in individual error subtypes.

The task itself limits which aphasic symptoms can appear. Single-word Cloze completions over a fixed vocabulary isolate word-choice errors — semantic, unrelated, and no-response — but cannot reproduce spoken-sound slips or made-up non-words, because the models lack a speech-production stage; formal, mixed, and neologism errors are ≤1% in the models versus about 20.5% in patients. Sentence-level fluency and pragmatics are not assessed, and automated scoring depends on operational rules such as top-5 accuracy matching and POS-gated GloVe similarity.

The pathway, not the lesion size, sets the curve

The paper's most decision-relevant result is a comparison, not a single number: at the same ablation fraction, the Transformer MLP pathway holds accuracy and then collapses late, the attention pathway plateaus early and then declines more smoothly, and the LSTM pathway declines steadily with grammar and meaning falling together. The authors argue this ordering survives all four fine-tuning regimes, which is why they attribute it to architecture rather than to recovery dynamics.

For readers evaluating this as a model of aphasia, the useful threshold is the one the authors emphasize: error-mix shifts toward less common content words appear from k = 0.1 in the LSTMs and Qwen MLP and from k = 0.2 in Qwen 1.8B attention, before overall accuracy falls far. That is the paper's case for looking at which words a system gets wrong, not only how often it is wrong.

The corresponding caution is equally specific. The clinical labels are assigned from a 100-prompt Cloze battery, the models differ in scale and corpus, and the ablation is uniform binary zeroing. The paper therefore supports architecture-dependent failure profiles as an in silico analogy for aphasic syndromes, with the MLP-versus-attention split as its clearest evidence — not a claim that any model has aphasia.

Sources used in this answer