Can nine-gaze photographs and deep learning diagnose strabismus accurately?

One photograph to screen, nine to subtype

The paper proposes a two-tier diagnostic system rather than a single model. Tier 1 is a screening model that reads only the primary-gaze photograph, the straight-ahead shot a person could take at home, and decides whether strabismus is present at all. Tier 2 is Strabismus-Net, which reads a full set of nine gaze photographs, one primary, four secondary and four tertiary positions, and assigns a subtype. The logic is that mass screening needs the cheapest possible input, while a preliminary clinical diagnosis needs enough views to separate exotropia, esotropia and vertical deviation.

The evidence base is described as the largest known dataset of strabismus gaze photographs to date. The retrospective set contained 84,897 photographs from 9,433 individuals, of whom 7,187 were diagnosed with strabismus and 2,246 were orthophoric, meaning their eyes were normally aligned. A separate prospective set of 10,458 photographs from 1,162 individuals was used for testing, with 848 strabismus cases and 314 orthotropia cases. Photographs came from inpatients and outpatients at a single ophthalmology department, captured with devices of at least 12 megapixels at distances of 33 cm to 1 m.

Why one view misses vertical deviation

The primary-gaze screening model performed well as a yes-or-no screen: AUC was 99.58% in internal validation and 99.53% on the prospective independent test. But when the same kind of primary-gaze model was asked to classify subtypes, the pattern changed sharply. AUC was 95.42% for orthotropia, 93.17% for exotropia and 84.28% for esotropia, but only 49.63% for vertical strabismus.

The authors attribute this gap to the geometry of the condition: some vertical deviations are not visible when the patient looks straight ahead and become apparent only in secondary or tertiary gaze positions. That explanation is the reason the second tier exists. It also means the 99.53% screening figure should not be read as evidence that one photograph can subtype strabismus; it answers a different question, presence versus absence, on a different input.

Cross-attention lets gaze positions inform each other

Strabismus-Net takes a batch of nine cropped eye-region images. A YOLOv8-based step first detects and crops the eye area from full-face photographs, which standardizes the input and removes identifiable face content. A pre-trained hybrid vision transformer then encodes each gaze image into a 512-dimensional vector.

The distinctive part is what happens next. A three-stage cascaded cross-attention module lets the nine gaze features interact, so the representation of one gaze position can be informed by the others through learnable query-key-value mappings. An adaptive feature aggregation step then computes importance scores for each position from the nine feature means and fuses them into one global 512-dimensional vector. Two classifiers share that vector: a three-way horizontal classifier (non-horizontal strabismus, exotropia, esotropia) and a binary vertical detector. In plain terms, the model does not average nine independent opinions; it learns which gaze directions matter for the decision at hand.

Each added module raised mean AUC

The ablation study isolates how much each design choice contributed, using mean AUC across the 10-fold cross-validation. A baseline vision transformer with simple feature concatenation reached 95.52%. Adding the dual-stream cross-attention mechanism raised it to 96.85%. Adding adaptive feature aggregation on top brought it to a peak mean AUC of 98.23%.

This is a within-paper comparison of model variants on the same data, so it supports the claim that the two modules improved performance in this setting. It does not establish that the same gain would transfer to another dataset or camera setup. The final architecture was trained with weighted categorical cross-entropy to handle class imbalance, and the fold with the best validation performance was selected for testing.

Strong subtypes, a weaker vertical category

On the prospective test set, Strabismus-Net reached AUCs of 97.15% for orthotropia, 99.19% for exotropia, 99.11% for esotropia and 90.20% for vertical strabismus. The ranking matters: the three categories that dominate the dataset are handled at close to ceiling, while vertical strabismus, the category the primary-gaze model could not detect at all, improves substantially but stays clearly behind.

The authors also report that the model's performance on certain rare vertical deviations remains suboptimal and that more granular, phenotype-specific diagnosis would require larger samples for rare etiologies. That limitation sits directly beside the vertical result, and it is the number a reader should weigh when judging whether nine-gaze imaging is sufficient for a given clinical use.

Comparable to seniors, better than juniors

Three junior ophthalmology residents and three senior strabismus specialists, each with at least 15 years of subspecialty experience, independently read the same prospective nine-gaze sets, blinded to clinical metadata. Pairwise AUC comparisons used DeLong's test. The model performed comparably to the three senior specialists across all subtypes, with P > 0.05.

The separation appeared where the task is hardest. For identifying vertical strabismus the model significantly outperformed the three junior residents (P < 0.001), and for confirming orthotropia it also outperformed them (P < 0.05). On the external test set the model's vertical strabismus AUC was 90.20%, against 75.93%, 77.20% and 78.80% for the three residents, while the residents were closer to the model on exotropia and esotropia. The comparison is against clinicians reading photographs only, not against clinicians using a full examination.

Attention lands on pupils, reflections and limbus

Grad-CAM heatmaps were used to visualize which regions drove the model's decisions. The authors report strong anatomical consistency between the model's attention and clinically relevant ocular features, specifically the pupil, corneal reflection and limbus. This is an interpretability check, not a performance measure, but it addresses a practical objection: whether the model is reading eye alignment or some incidental image artifact.

The paper also embeds both tiers in an integrated management platform intended to standardize data collection and streamline referrals to strabismus subspecialists. The authors state that the platform's real-world implementation and referral outcome benefits have not yet been thoroughly tested and that prospective, multi-center trials are needed. The reported results therefore concern diagnostic accuracy on photographs, not improved patient outcomes from deploying the workflow.

Single-center data and excluded photographs

Several boundaries qualify the numbers. All photographs were taken at one department in China, and the datasets are not publicly available, so the reported AUCs are internal to this setting and this imaging protocol. Photograph sets were excluded if they showed motion blur beyond a 2-pixel edge sharpness threshold, illumination variance above 30% across images, or if the subject wore spectacles; sets also required corneal exposure above 80% and gaze deviation under roughly 10 degrees from the target direction. The model was therefore evaluated on images that passed a quality filter, not on whatever a home user might submit.

The practical priority follows from the comparison rather than from the headline AUC. The screening tier is the piece suited to broad triage, and the nine-gaze tier is the piece that adds value precisely where a single straight-ahead view fails: vertical deviation and confirmation of normal alignment, the two scenarios where it separated from junior residents. Reading the 99.53% screening figure as a diagnosis, or the 90.20% vertical figure as solved, would misplace both results.

Sources used in this answer