Can ownership routing separate interleaved text sources without regenerating words?

The problem is ownership, not missing words

Word-level text unmixing starts from a case where the lexical evidence is complete but the attribution is gone. Multiple sources get serialized into one stream: overlapping speech transcripts merge chronologically, document extraction flattens two-dimensional layouts into a wrong linear order, and concurrent agents or unlabelled event logs emit partial outputs into a shared stream where only arrival order survives. The words are all there; what is unknown is which source each word occurrence belongs to.

The paper formalizes the task as recovering the distinct source sequences given the interleaved stream and a known source count K, such that every observed word occurrence is used exactly once and its original within-source order is preserved. That is a finer target than utterance-level conversation disentanglement, paragraph-level style-change detection, or temporally grounded speaker diarization, because ownership can shift between adjacent words and must be resolved from the mixed lexical stream alone.

Why direct generation breaks exact reconstruction

Prompting an LLM to emit separated source arrays couples two different jobs: deciding ownership and producing words. The paper argues this creates two failure modes. Each ownership assignment changes the partial source contexts needed to interpret later words, so routing must be history-conditioned. And free generation can omit, duplicate, or alter observed words, which directly violates the exact-reconstruction objective even when the underlying ownership deduction was structurally correct.

EPOR separates the two. A causal language model is adapted to predict a single ownership label per observed word, conditioned on the interleaved stream and prior routing decisions. Reconstruction is then deterministic: indexed reconstruction mechanically gathers the original word occurrences according to the predicted routes. The model never writes the words it is sorting.

Constrained decoding keeps routes structurally valid

At inference, EPOR uses completion-safe constrained beam search. The admissibility rule restricts each next label to actions that still admit a complete canonical route: after choosing a label, the number of remaining positions must be at least the number of unseen sources. EOS is excluded from the route alphabet, and decoding stops after exactly T ownership symbols, where T is the number of observed word occurrences.

This guarantees structural validity, not correctness. Every accepted route is a complete canonical partition, and reading each label set in increasing index uses every input position once while preserving order within each assigned source. But finite-beam search only approximates maximization of the route score; it does not enumerate the full support or guarantee a globally optimal route. The paper states that beam pruning cannot make a retained prefix invalid, yet it can discard the globally highest-scoring route.

UNMIXBENCH covers five evaluation tracks

The benchmark spans controlled synthetic mixtures, timestamp-derived speech from AMI and ICSI, layout-derived document streams from ReadingBank, and simulated concurrent digital outputs. The serialization extension contains 3,433 evaluation views. Its 1,200 digital views are three schedules over 400 shared source groups, so group-level inference keeps the schedules together. The ReadingBank view contains 833 examples from 828 test documents.

The tracks differ in how mixtures were produced. AMI uses annotated word-time merges over 1,000 windows from 59 meetings with mean mixing intensity 0.275; ICSI development uses 400 windows from 15 meetings at 0.290; ReadingBank uses metadata-derived reading flows at 0.176; and the digital smooth, bursty, and unequal-rate profiles sit at 0.582, 0.266, and 0.531. Because these tracks use different serialization mechanisms and different mean intensities, their error rates are not interchangeable measures of the same difficulty.

The 4B model leads finetuned baselines by 22.3%

Across the five principal UNMIXBENCH tracks, a 4B EPOR model achieves the lowest mean minimum-permutation word error rate among finetuned baselines, reducing the five-track mean by 22.3% relative to compact source-array generation, and remains competitive with zero-shot frontier LLMs. Minimum-permutation word error rate is computed against gold sources after choosing the best global relabeling, with the denominator being the total gold word count, so it measures word-level disagreement rather than whether a whole source was assigned the right identity.

The comparison is against task-finetuned generators and structured baselines, not against every possible system. The paper also reports a matched representation control, PI-Route, which replaces canonical targets with arbitrary source labels and enumerates all K! global relabelings; it matches EPOR in backbone, trainable parameters, data, learning rate, epochs, context length, source counts, training runs, beam budget, reconstruction map, and metrics. That control isolates how much of the gain comes from canonical first-appearance routing rather than from extra capacity.

Training setup and what the model sees

The task-adapted comparisons use post-trained Qwen3.5-4B with NF4 quantization, rank-32 QLoRA, alpha 64, dropout 0.05, two epochs, effective batch size 16, and three training runs. EPOR and PI-Route use learning rate 2e-4 and a context of 1,024 LM tokens. A length audit finds that 3.4% of supervised EPOR sequences exceed 512 LM tokens and none exceeds 1,024.

The 10,000-example training set draws K from (2,3,4) with probabilities (0.6,0.3,0.1), realized as 5,955, 2,998, and 1,047 examples, consuming 25,092 source spans without reuse. Input words are indexed and JSON-encoded before the model-specific chat template is applied. The prompt asks for exactly T single-digit labels, one per indexed input token, with the first label fixed to 0 and new labels introduced only as one plus the largest previous label. The prompt is loss-masked; route symbols and the assistant termination suffix are supervised.

What the identifiability audit does and does not show

Because a legal partition can still be semantically wrong, the paper ran a two-stage audit on a stratified panel of 120 controlled-MultiNLI examples spanning K in (2,3,4), four source-similarity bands, and five mixing settings. A GPT-5.6 Sol reviewer agent was shown the mixed text, K, and the recorded source partition and asked to construct a materially different legal partition that remained linguistically plausible. It found no clear alternative for 103 examples, marked seven uncertain, and flagged three as plausible alternatives and seven as strongly underdetermined.

A human reviewer then examined all 120 examples and found no materially different plausible alternative for the 103 initially clear cases. Of the 17 non-clear cases, only two K=4 examples, at target mixing intensity 0.75 and 1.0, retained a plausible alternative; the other 15 were judged to have one defensible partition. The paper is explicit that the panel was designed to probe difficult cases, so this descriptive rate is neither an unbiased prevalence estimate nor a human ceiling on minimum-permutation word error rate.

Where these guarantees stop

Three boundaries matter for interpreting the result. The formulation assumes the source count K is known; jointly inferring the number of sources and their ownership is left as an extension. Finite-beam decoding guarantees structural validity but not correct ownership, and the paper's candidate-ranking analysis indicates substantial ranking error remains even among legal routes, so better route scoring or reranking could still yield gains. And the formulation isolates ownership recovery under complete lexical observations: real pipelines may also contain ASR or OCR insertions, deletions, or substitutions, so extending evidence-preserving routing to noisy or incomplete observations remains open.

The practical priority follows from that split. When every word occurrence is observed and only ownership is missing, decoupling ownership inference from lexical regeneration is the reliable alternative to direct generation, and the 22.3% five-track mean reduction over compact source-array generation is the number to weigh. When words themselves may be missing or corrupted, or when K must be inferred, the exact-reconstruction guarantee no longer applies and the paper's evidence does not cover the case.

Sources used in this answer