Can a soft-prompt time-series language model run privately on a smartphone?

A deployment result, not a clinical validation

The paper asks whether a time-series language model can be moved from a server onto a phone without giving up the temporal structure of the sensor signal. Its answer is a working port: the encoder, projector, and language model all execute locally on Apple silicon through the MLX-Swift runtime, and the authors state that no health data leaves the device during inference.

That claim is deliberately narrow. The reported results cover implementation parity and on-device context economics; the authors say explicitly that they do not test clinical performance or patient-facing deployment. The benchmarks use fixed, public, de-identified samples from Sleep-EDF and ECG-QA, not prospective HealthKit records from patients. So the finding is that this pipeline can run privately on a phone, not that its answers are clinically reliable in new settings.

Why embeddings beat serialized samples

A time-series language model couples a time-series encoder to a language model so the model can produce natural-language descriptions from raw physiological signals. In the soft-prompt variant, a raw series is split into fixed-length patches, a CNN patch encoder and a Transformer turn each patch into one embedding, and a learned projector maps those embeddings into the input space of a frozen Llama-3.2-1B backbone. The projected vectors are interleaved with the embedded text prompt as soft-prompt tokens — continuous vectors injected directly into the model's embedding sequence rather than vocabulary tokens.

The alternative is text serialization: converting raw samples into tokens the language model already understands. That keeps the standard generation path but spends context on the signal itself. The paper's benchmark format charges one soft-prompt token per four-sample patch, rounding up partial patches, against 3 BPE tokens per raw sample for text. That is the mechanism behind the roughly 12-fold token gap, and it is why the soft-prompt path leaves more of the phone's limited context and memory for the actual reasoning task.

The port reproduces the reference outputs

Moving the model required two stages: PyTorch to Python MLX, then Python MLX to Swift MLX. Staging the migration this way separates framework-level differences from errors introduced in the Swift port. The authors also had to rebuild parts of the generation path, because MLX-Swift's high-level generation accepts token identifiers rather than precomputed input embeddings.

Parity was checked numerically. The encoder, projector, fused encoder-projector pipeline, and rotary embedding matched the Python MLX reference within a maximum element-wise tolerance of 10⁻⁴ and mean 10⁻⁵, with 10⁻³ for the rotary embedding, on the CPU backend. End to end, the Python MLX and Swift implementations produced identical greedy-decoded token sequences. Against the PyTorch reference, decoded answers agreed, although intervening rationale wording could differ under bf16 arithmetic. This matters because the whole privacy argument depends on the phone running the same model, not an approximation of it.

Twenty sleep clips agree on every answer

Parity of tokens is not the same as task accuracy, so the authors also ran the first 20 samples of the Sleep-EDF chain-of-thought test split through the PyTorch reference, the Python MLX reference, and the Swift port on the iPhone. All three scored 17/20 and agreed on every prediction, including the same three errors with the same incorrect labels. Agreement on the errors is the stronger signal: a port that only approximated the reference would be expected to diverge first where the model is least confident.

The measured 85% is consistent with the 81.08% reported by Langer et al. on the full test split, but 20 samples give a 95% interval of roughly ±16 points. The authors therefore treat this as corroboration of the deployment rather than an accuracy estimate. They also note that soft prompting does not appear to trade quality for context economy: on this task the reference reports 69.88 F1 against 9.05 for the same backbone fine-tuned on a text serialization of the same signal.

The 12-lead ECG case decides feasibility

The token gap becomes decisive when channel count rises. Both encodings are charged per lead, so a 12-lead ECG window of N samples costs 36N text tokens versus 12⌈N/4⌉ soft-prompt tokens. At L=100 samples per lead, text serialization already needs 4,133 prompt tokens and 4.3 GiB; every longer window (L≥250, at least 9,533 tokens) was skipped to avoid an uncatchable operating-system memory termination.

Soft prompts ran the full evaluated 1,000-sample clip with time to first token at or below 11.6 s and peak memory at or below 4.2 GiB. The bf16 backbone weights alone account for about 2.3 GiB of that peak, with the remainder growing with prompt length. The practical cutoff in the measured sweep was the lower 4,300-token on-device prompt budget, used to preserve runtime memory headroom during generation.

Single-channel EEG shows the same mechanism more gently. Text survives to L=1000 but prefills about 7 times the tokens of the soft-prompt arm, so its time to first token is roughly 6 times larger (14.8 ± 0.8 s versus 2.4 ± 0.1 s at L=1000), and it still exceeds the budget at the full original 1,500-sample window. Soft-prompt time to first token stayed at or below 3.0 s and peak memory at or below 3.9 GiB at every length.

What the latency numbers do not settle

The latency and memory figures are means over five repeated sweeps, so they describe repeated-run performance rather than an isolated cold start or a verified steady state. For EEG, the spread is under 6% of the mean at L≥1000 but reaches roughly 30% at L≤500, where short prefills are dominated by fixed overhead rather than compute. For ECG, the spread is 19–27% at every length, driven by a faster first run.

Thermal state also moved during testing: for EEG it rose from nominal to serious within the first sweep and stayed there, while for ECG it read nominal at each run start even though the first run was 1.6 times faster than the last. Thermal throttling is a possible explanation for the ECG slowdown, but the authors say it was not directly established. They do not report energy consumption. At truncated lengths the soft-prompt arm's generated text degenerates, because a window shorter than the 1,000-sample training clip is out of distribution; the reported token, latency, and memory costs are unaffected because they depend on the input encoding rather than the decoded content.

Where the deployment claim stops

The compression ratio is fixed rather than tuned. Patch length and stride follow the OpenTSLM reference, and the encoder, projector, and LoRA adapters are loaded from its checkpoints; varying the ratio would require retraining all three, which the authors describe as a change to the model rather than to its deployment. The two LoRA adapters, each roughly 1% of the model's parameters, adapt the frozen backbone to the ECG and EEG tasks, and the selected adapter determines the modality-specific mapping at inference.

The authors frame PocketTSLM as an implementation and deployment contribution, not as evidence of clinical performance in new settings. The main remaining step they identify is a prospective evaluation on real-world wearable time-series data with predefined review, safety, monitoring, and governance procedures. The study used public, de-identified research datasets and did not involve prospective recruitment, participant interaction, intervention, or collection of identifiable private information.

The priority is context headroom

For anyone weighing on-device physiological reasoning, the paper's most decision-relevant comparison is not accuracy but what fits. Extrapolated to the backbone's 8,192-token context window, text supports about 2,659 single-channel EEG samples (roughly 53 s of 50 Hz data) but only about 212 samples per ECG lead, whereas soft prompts support about 31,912 and 2,553 respectively under the same budget.

That is the threshold to carry away: the soft-prompt path is what makes multi-lead, longer-window physiological input feasible on a phone at all, and the 12-lead ECG case is where the text path fails first. The 20-sample sleep-staging agreement anchors the port to the original PyTorch implementation, but it is a parity check, not a clinical accuracy claim. The next question the paper leaves open is whether this deployment holds up on prospective wearable data under real review and safety procedures.

Sources used in this answer