Do bilingual language models capture L1-driven biases in L2 reading times?

Why a monolingual model cannot see L1 transfer

Surprisal theory ties the effort of processing a word to how unpredictable that word is in context. Monolingual language-model surprisal is a well-established predictor of native-language reading times, and the paper starts from the observation that such a model is trained only on the target language. If a reader's first language actively shapes how they read a second language, a model that has never seen that first language cannot, by construction, represent that influence.

The study therefore asks a narrower question than whether language models predict reading in general. It asks whether the cross-lingual transfer inside a bilingual model lines up with the L1-driven biases that human readers bring to L2 reading. The test is whether matching the model's L1 to the reader's L1 buys predictive accuracy that a monolingual English baseline and L1-mismatched bilingual models do not provide.

Four eye-tracking measures separate early and late processing

The paper evaluates surprisal against four eye-tracking measures, and the distinction matters for reading the results. First fixation duration is the length of the first fixation on a word. Gaze duration sums all fixations on a word before the eye first leaves it. Both are treated as early measures reflecting initial lexical access, such as word-form recognition and meaning retrieval.

Go-past time runs from the first fixation on a word until the gaze moves to its right, including time spent regressing to earlier text. Total fixation duration sums every fixation on a word, including later returns to it. These late measures are thought to reflect post-lexical work such as structural reanalysis and syntactic or semantic integration. Because the two families of measures index different processing stages, a model advantage in one cannot be substituted for an advantage in the other.

Matched and mismatched models isolate L1-specific transfer

The authors trained bilingual GPT-2 models across multiple Steps of Exposure (SoE) stages, varying when L2 English is introduced during L1 learning, so that both the duration of initial L1 training and the amount of L2 exposure are manipulated. For each of 11 L1-L2 language pairs they trained four SoE bilingual models plus a monolingual English baseline, and derived word surprisal as negative log-probability, summing subword surprisals for words split by the tokenizer.

Evaluation used the Multilingual Eye Movement Corpus (MECO L2), in which participants from 13 L1 backgrounds read the same 12 English texts. The authors kept each institution as a distinct subset rather than pooling by L1, excluded the Basque subset for insufficient representation in FineWeb-v2, and added the native English (En-UK) subset as a control. A bilingual model counts as matched when its L1 equals the reader subset's native language; all other bilingual models are mismatched.

Predictions were tested with linear mixed-effects models using crossed random intercepts for participants and words, with log-frequency, word length, word position, and the preceding word's log-frequency and length as baseline covariates. Two comparisons carry the argument: bilingual versus monolingual, measured as the change in pseudo-R² over a surface-features baseline, and matched versus mismatched, measured as a difference in log-likelihood and, in an encompassing model containing both surprisal predictors, as the unique variance added by matched surprisal via a likelihood ratio test.

The L1 advantage sits in late reading measures

Matching the reader's L1 improved prediction of L2 reading times, and the abstract reports that this predictive advantage is concentrated in the late-stage eye-tracking measures. That localization is the paper's most informative result: the gain appears where post-lexical integration is assumed to happen, not in the earliest word-recognition measures.

The native English control group serves a specific purpose here. It checks whether a bilingual model simply fits reading times better in general; if the advantage were a generic multilingual improvement, it should not be specific to L2 reading. The paper reports that its two evaluation frameworks, bilingual versus monolingual and matched versus mismatched, yield consistent conclusions, which is why the authors read the pattern as evidence of L1-specific transfer rather than a broad multilingual effect.

Script overlap and proficiency bound the effect

Two reader-side variables modulate the advantage. The first is L1-L2 script overlap: the benefit of matching varies with whether the reader's L1 shares a writing system with English. The second is proficiency, measured with LexTALE, where the advantage peaks among intermediate-proficiency readers rather than being uniform across the sample.

These modulations change how the result should be used. A claim that bilingual surprisal captures cross-linguistic influence is not a claim that it helps equally for every reader or every language pair. The paper's own framing places the strongest signal at intermediate proficiency and treats script overlap as one axis of variation, so the appropriate reading is a conditional advantage rather than a constant one.

Evidence for transfer, not a full account of L2 reading

The authors conclude that bilingual language models capture aspects of cross-linguistic influence and that such influence is present in L2 reading behavior. The word aspects matters. The finding is that matched-L1 surprisal explains variance beyond both a monolingual baseline and mismatched bilingual surprisal within the same regression analysis, which is a statement about predictive alignment, not a demonstration that the model reproduces the reader's cognitive mechanism.

The study's scope is also explicit. Evaluation is restricted to English as the target L2 and to a single eye-tracking corpus, leaving open whether the pattern generalizes to target languages with very different typology, such as highly agglutinative languages or ones with freer word order. Script overlap is only one axis of typological variation; word-order alignment and morphological similarity are named as factors likely to matter but not tested here. Some MECO subsets were collected in regions where regional varieties are spoken, while the models were trained on standard general-domain corpora, a mismatch the authors note could add noise to individual subset results. Proficiency is operationalized through LexTALE, a lexical measure whose validity as a global proficiency index has been questioned. Finally, each model configuration used a single initialization seed, so variance across weight initializations was not measured; the authors argue robustness from the breadth of the setup instead, across four SoE models for each of 11 L1s compared against monolingual baselines.

Match the L1 before adding model scale

For anyone using language-model surprisal to model non-native reading, the paper's priority is specific: the L1 of the model should match the L1 of the readers being modeled, and the evaluation should look at late measures rather than only early ones. A monolingual English model, or a bilingual model trained on an unrelated L1, is the wrong comparison point if the goal is to capture L1-driven reading behavior.

The result also sets a threshold for expectations. The advantage is largest for intermediate-proficiency readers and depends on script overlap, so a null or weak effect in a sample skewed toward advanced readers or a particular script pairing would not by itself contradict the finding. The most defensible takeaway is conditional: matched-L1 surprisal adds explanatory variance in late L2 reading measures, and that is where the claim should be tested and where it should be trusted.

Sources used in this answer