Does Phenomenon-Graph JEPA pretraining improve label efficiency in cardiorespiratory sensing?

A label-efficiency test with a built-in graph control

Contactless sensing with millimeter-wave radar and RGB-D cameras can record cardiac and respiratory waveforms continuously, but labeled recordings are scarce because each label requires a supervised acquisition session. The paper asks whether self-supervised pretraining on unlabeled signals can make a later classifier accurate with fewer labeled episodes. Its second question is narrower: when a recording contains both cardiac and respiratory streams, does it help to restrict prediction to streams that measure the same physiological phenomenon?

The proposed model, Phenomenon-Graph JEPA, is a joint-embedding predictive architecture. Instead of contrasting augmented views or pushing negative pairs apart, it predicts the stopped embeddings of target windows. The base configuration uses four processed one-dimensional streams and no negative pairs or synthetic augmentation. Each stream passes through a temporal convolutional branch and a band-limited spectral branch, producing 192 features per stream and 768 features across the four OMuSense streams, with no learned fusion layer.

The graph question is treated as a hypothesis rather than an assumption. A typed graph links radar cardiac to camera remote photoplethysmography and radar respiration to depth respiration. A wrong-edge graph crosses the phenomena while keeping the same number of edges and directed predictors, and an all-pairs graph adds all six links. The typed and wrong graphs each train 2.19 million pretraining parameters, so that comparison changes topology alone; the all-pairs graph trains 3.19 million and therefore also changes predictor capacity.

Prediction targets replace negative pairs

The motivation for avoiding contrastive learning is specific to cardiorespiratory data. Contrastive methods need transformations that preserve the downstream class and a rule for declaring which window pairs are negative. For these signals, time dilation changes breathing rate, amplitude scaling changes a volume proxy, and two windows separated in time can describe the same physiological state. Those conditions make both design decisions uncertain.

Phenomenon-Graph JEPA instead trains three prediction losses. Cross-edge prediction maps a combined online projection from one stream to the temporal target of a linked stream at the same time. A separate spectral predictor maps between spectral projections. A within-stream predictor forecasts a later window of the same episode, using a sinusoidal encoding of the displacement between context and future window. Targets come from exponential-moving-average copies of the encoders and projectors with a stop-gradient, so they never provide inputs to the predictors.

Because a constant embedding would minimize every cosine term, variance and covariance penalties act on the online projections to keep dimensions active and non-duplicative. One detail matters for interpreting the label-efficiency claim: the loss uses no class targets, but annotation-derived episode boundaries constrain which future windows are sampled. The pretraining pipeline is therefore free of class labels, not of annotation information.

The gain is largest at the smallest label budgets

The main evidence comes from OMuSense-23 activity recognition, evaluated on the same ten fold-V test participants. Under the base configuration, pretraining improves the normalized log-budget area over matched supervised training by 3.91 percentage points, with a 95% participant bootstrap interval of 2.08 to 5.80 and a Holm-adjusted p of 0.006. The improvement favors JEPA for all ten participants.

The per-budget curve shows where that area comes from. Under the base configuration, pretraining is ahead of supervision at all five budgets, and the gap is largest at the lowest budgets: 7.6 points at 1.11% labels and 6.1 points at 4.44%, narrowing to 1.4 points with full labels. Macro-F1 is 36.3 versus 28.7 at nominal 1% labels and 75.1 versus 73.7 at full labels.

The second configuration, which adds shift and sensor masking, gives 3.74 points for the same participants, with an interval of 1.68 to 5.78 and a Holm-adjusted p of 0.0234. Its curve is not uniformly better: pretraining leads at 1.11%, 4.44% and 10% labels, while supervision leads at 50% and 100%, reaching 78.6 versus 76.5 macro-F1 at full labels. The paper's claim is therefore about label efficiency measured by the area, not a universal improvement at full supervision.

Typed physiological edges do not separate from controls

The graph hypothesis does not receive the same support. Typed edges minus wrong edges gives −0.74 percentage points under the base configuration, with a 95% interval of −1.44 to 0.03 and a Holm-adjusted p of 0.2031. Typed edges minus all pairs gives −0.43 points, with an interval of −1.65 to 0.87 and a Holm-adjusted p of 0.5352. Neither control establishes a benefit from physiological typing.

The per-budget numbers make the same point less abstractly. At 1.11% labels, the wrong-edge graph reaches 38.52 macro-F1 and the all-pairs graph 36.63, against 36.25 for the typed graph. At 4.44%, the wrong-edge graph reaches 48.72, above both the typed graph at 45.44 and the all-pairs graph at 47.23. The typed graph is not the best graph at the budgets where pretraining helps most.

This is why the paper separates two claims that could easily be conflated. The measured benefit belongs to the predictive learning framework under the evaluated conditions. The physiological prior used to organize training remains an unresolved contributor, and the authors state that the observed gains do not establish that the proposed graph structure accounts for them.

Published fusion and prior self-supervision remain separate baselines

The paper also compares against published fusion results on the same corpus, and published fusion remains stronger. Those fixed-split scores match Fold I in a five-fold table, which also reports 56.40 for sex and 79.20 for joint pose–activity recognition, different endpoints. Because the fixed split cannot be identified with fold V from the printed fold names alone, the paper does not treat the two as the same held-out cohort.

A historical contrastive self-supervised reimplementation is likewise not a matched control. It reaches 72.71 activity accuracy against 74.00 reported originally, but preprocessing, adaptation, seed counts and aggregation prevent objective-only comparisons. Its macro-F1 curve is below base pretraining at 1.11%, 50% and 100% labels but above it at 4.44% and 10%. Because adaptation, representation width and seed count differ, those crossings describe two systems rather than a controlled comparison of objectives.

Accuracy and macro-F1 also diverge here. At nominal 1% labels, JEPA and supervised accuracy are 38.25 versus 33.78 in the base configuration and 40.95 versus 34.14 in the second. Accuracy and balanced accuracy coincide on these class-balanced participant windows, whereas macro-F1 differs, so those accuracy values are not interchangeable with the macro-F1 values in the main table.

The advantage does not carry to wrist-only WESAD

Two completed WESAD meditation protocols test whether a corresponding label-efficiency advantage appears outside the contactless setting. Both are wrist-only at test time: pretraining uses wrist blood volume pulse paired with chest ECG and wrist electrodermal activity paired with chest electrodermal activity, then discards the chest encoders. Neither protocol establishes a pretraining advantage. Rotations give +0.25 percentage points for JEPA minus supervision, with a 95% interval of −3.88 to 4.62 and a Holm-adjusted p of 0.9129; leave-one-subject-out gives −1.16 points, with an interval of −4.39 to 1.56 and a Holm-adjusted p of 0.5054.

The paper explicitly declines to read this contrast as a sensor-contact effect. WESAD and OMuSense differ in task, label budgets, participants, window lengths and training schedules, so their difference does not isolate contact as a causal factor. The WESAD result is better read as a boundary on where the OMuSense label-efficiency finding has been demonstrated, not as evidence about why it fails elsewhere.

An optional Takens-inspired delay-coordinate variant adds a further partial result. It improves a validation comparison against a learned-history control with matched temporal support, but it was not included in the main test evaluations. That keeps it outside the headline label-efficiency claim.

Trust the label-efficiency area, not the graph story

For a reader deciding what to take from this paper, the priority is the split between the two outcomes. The label-efficiency area gain is the result with a matched supervised baseline, a stated interval, a corrected p-value and consistency across all ten participants under the base configuration. The typed-graph advantage is the result that failed its own capacity-matched and denser controls, and the wrong-edge graph was actually the strongest of the three graphs at the two lowest budgets.

The conditions that most constrain the positive result are stated by the authors. All OMuSense comparisons use the same ten test participants, who also appeared in earlier project evaluations, and the second configuration was fixed after the first had been evaluated. The two positive results are therefore observations under two configurations on one cohort rather than independent replications, and the Holm families do not correct every exploratory comparison made across the complete project.

The practical reading is a scenario split rather than a general recommendation. Where labeled episodes are very scarce, the base configuration's advantage over matched supervision is largest, at 7.6 points at 1.11% labels and 6.1 points at 4.44%. Where full labels are available, the second configuration's supervised baseline is ahead at 78.6 versus 76.5 macro-F1. The next test the paper itself points to is comparison against stronger feature-fusion baselines under matched protocols, together with generalization to independent cohorts, because the graph prior and the delay coordinates remain unproven contributors rather than established sources of the gain.

Sources used in this answer