Can self-supervised learning classify thyroid cytology images with fewer labels?

Fewer expert labels for eight lesion types

Thyroid fine-needle aspiration cytology is used to evaluate thyroid nodules before surgery, and building deep learning models for it normally requires many images labelled by experts. The paper asks whether self-supervised learning can reduce that labelling burden without giving up classification accuracy.

The authors retrospectively analysed cytology images from 393 thyroid nodules and extracted 142,158 image patches. The classification target was not simply benign versus malignant: the models had to assign patches to eight lesion types, including follicular neoplasm, oncocytic follicular neoplasm, papillary thyroid carcinoma, poorly differentiated thyroid carcinoma, anaplastic thyroid carcinoma, medullary thyroid carcinoma, primary thyroid lymphoma and non-neoplastic lesion.

Pretraining on unlabelled patches, then fine-tuning

Self-supervised learning here means training a model to build useful image representations from unlabelled images, before any diagnostic labels are used. Three such methods — DINOv1, iBOT and MAE — were used to pretrain vision transformer models on the unlabelled patches. Only afterwards did the models undergo classification training with different amounts of labelled data.

The comparison baseline was a vision transformer of the same family pretrained on ImageNet, the general object-image dataset. This matters because it isolates the value of pretraining on cytology images without labels, rather than the value of using a vision transformer at all. Performance was measured with five-fold cross-validation, precision–recall area under the curve (PRAUC) and macro-averaged F1, which averages the F1 score across the eight classes rather than letting common classes dominate.

At 1% of labels, self-supervision holds up

The gap between pretraining strategies widened as labels shrank. With 100% of labelled data, PRAUC was 0.9302 for ImageNet pretraining, 0.9500 for DINOv1 and 0.9542 for iBOT. At 10%, the values were 0.8644, 0.9321 and 0.9418. At 1%, ImageNet pretraining fell to 0.7065, while DINOv1 reached 0.8711 and iBOT reached 0.8987.

Macro-averaged F1 followed the same pattern: at 1% of labels, ImageNet pretraining scored 0.6565, DINOv1 0.7932 and iBOT 0.8060. MAE was the exception among the self-supervised methods, scoring 0.7620 PRAUC and 0.7396 F1 at 1%, below the other two self-supervised models and closer to the ImageNet baseline. So the paper's claim is not that any self-supervised method works equally well; the benefit depended on which pretraining objective was used.

iBOT scored higher, DINOv1 was chosen

iBOT produced the best downstream scores in the low-label setting, yet the authors selected DINOv1 as the primary approach. Their stated reasons are practical: iBOT uses a dual loss at both token and global levels, making training more complex and requiring more elaborate parameter tuning, and its attention maps frequently showed artefacts that hindered identification of focus areas. DINOv1 was easier to train and to visualise.

This is an author judgement about usability, not a performance result. Readers comparing the two methods should keep the distinction: iBOT had the higher PRAUC and F1 at 1% of labels, while DINOv1 was the model carried forward into the diagnostic pipeline and the comparison with cytotechnologists.

The pipeline outperformed four cytotechnologists

The diagnostic pipeline, built on the DINOv1-pretrained model with LightGBM, reached an average class-specific accuracy of 0.94. In the same comparison, the four participating cytotechnologists (labelled A to D) had mean accuracies of 0.74, 0.61, 0.65 and 0.48, and their majority vote reached 0.70. The AI's mean F1 was 0.94 and mean PRAUC 0.99.

The per-class breakdown shows where human and model performance diverged most. For poorly differentiated thyroid carcinoma, F1 was 0.02, 0.02, 0.05 and 0.23 for the four cytotechnologists, versus 0.96 for the AI. The authors offer explanations rather than proof: the diagnosed samples came from a different institution with slight differences in sample preparation and staining, human diagnosis normally uses the whole specimen rather than a single field of view, and about 90% of thyroid malignancies are papillary carcinomas, so experts rarely encounter the other cancer types.

Combining humans and AI improved on the human majority vote: the four-person majority vote plus AI reached mean accuracy 0.79 and mean F1 0.72, compared with 0.70 and 0.63 for the four-person vote alone. The authors also state that AI classified lesions more than 100 times faster than humans.

Lesion-specific clusters formed without diagnostic labels

When the DINOv1-trained model's image features were projected into two dimensions with t-SNE, clusters corresponded to lesion types even though no supervised labels were used during that representation learning. The authors report that the representations appeared to capture similarities in colour and shape and suggested the potential to separate different cytological features of thyroid carcinomas.

This is presented as an observation about the learned representations, not as a validated diagnostic capability. The paper frames it as a hint that such models might be interpretable without additional supervised tasks and that the representations could potentially be repurposed for other downstream tasks such as similar image retrieval, segmentation and prognosis prediction. Those applications were not tested here.

Tumour cells at patch edges were misread

Error analysis of the model trained on the self-supervised vision transformer found that images with tumour cells at the edges were often misidentified as non-neoplastic lesions. The authors attribute this to a mismatch in how images are handled: human annotators work from original microscopic images before dividing them into patches, so they can recognise the entire tumour cell area, whereas the model is trained and predicts only on individual patches.

This is a concrete boundary of the patch-based pipeline rather than a general failure of self-supervised learning. It also indicates where additional work would be needed before such a system could be used on whole microscopic fields.

One institution limits generalisability

The study was confined to a single institution, which the authors say limits their ability to assess generalisability across diverse patient populations and imaging protocols. They call for applying self-supervised learning to multi-institutional cytology image datasets. The human-versus-AI comparison carries an additional caveat noted by the authors: the cytotechnologists were diagnosing samples that differed from those at their own institutions.

Within that scope, the paper's most decision-relevant comparison is not self-supervision versus supervised learning in general, but which pretraining objective to use when labels are scarce. At 1% of labelled data, iBOT and DINOv1 both stayed above 0.87 PRAUC while ImageNet pretraining dropped to 0.7065, and MAE did not deliver the same benefit. The practical priority the evidence supports is choosing a self-supervised objective suited to cytology images and validating it across institutions before treating the reported accuracy as transferable.

Sources used in this answer