Can reference-based color harmonization improve cross-dataset endoscopic image analysis?

What the harmonization actually changes in each image

The method targets one specific cause of cross-dataset failure: the shift in global appearance, especially color distributions, that arises when images come from different endoscope sensors and light sources. It does not attempt to fix anatomy, texture, or lesion appearance.

Concretely, the training dataset is treated as the reference. All its images are converted from RGB to CIE-Lab, and channel-wise reference means and standard deviations are computed once per channel (L, a, b) using pixels aggregated across the full reference dataset. These reference statistics stay fixed for the whole experiment.

Each target image is then converted to Lab, its own per-image channel means and standard deviations are computed, and a channel-wise affine mapping aligns the target distribution to the reference distribution. A small constant of 1e-6 is added to standard deviation terms to avoid division by zero, values are clipped to valid CIE-Lab ranges, and the result is converted back to RGB and saved under the original file name so existing label files still match.

Because the transformation is deterministic and applied offline as preprocessing, the architecture, training procedure, learned weights, inference settings, and decision threshold all stay unchanged. That design is what lets the authors attribute any performance change to input appearance rather than to retraining or parameter tuning.

Why CIE-Lab separates color from brightness

CIE-Lab separates luminance from chromatic components, which matters here because the paper's stated reason for choosing it is that it isolates red and yellow hues well, and those hues dominate abnormalities in endoscopic images.

This is also why the choice of color space is not incidental. The paper evaluates Z-score normalization variants in HSV, YCbCr, Lab-L, and Lab-ab as lightweight ablation baselines. The Lab-L variant normalizes only the luminance channel, while Lab-ab normalizes only the chromatic channels, so those ablations probe whether brightness alignment or color alignment is doing the work.

The proposed method differs from the compared appearance-transfer approaches in how much manual choice it requires. FDA, PCA/SVD, and palette-based color transfer need manual decisions such as spectral region size, transformation strength, color space, or palette size; the proposed method instead uses fixed CIE-Lab statistics computed from a reference dataset and applies the same deterministic transformation to every target image.

How the cross-dataset evaluation was organized

Models were trained once on a reference dataset and then tested on independent target datasets acquired under different imaging conditions. Experiments were split into two modality-consistent tracks.

In the flexible colonoscopy track, DS1 (LDPolyp) served as the reference, with DS2 (EDD2020) and DS3 (KUMC) as targets. In the capsule endoscopy track, DS4 (Kvasir-Capsule) served as the reference and DS5 (KID) as the target. The target sets contained 127 polyp and 53 normal images (DS2), 103 polyp and 30 normal images (DS3), and 44 polyp and 35 normal images (DS5).

Performance was measured with sensitivity, specificity, F1-score at a threshold of 0.5, and AUC. The paper emphasizes AUC because it reflects probability-ranking ability and is comparatively less sensitive to class imbalance than the threshold-dependent metrics. YOLOv7 was used as the real-time detection backbone for polyp localization, while ResNet50 and InceptionV3 were used as classification backbones to check whether the benefit held across architectures.

For image-level evaluation, an image counted as positive if at least one polyp-class detection survived non-maximum suppression, and the maximum detection confidence score served as the image-level prediction score.

The largest gain appeared on the hardest transfer

On DS3, with DS1 as reference, the proposed harmonization produced the strongest and most consistent results across all three backbones. For YOLOv7 it reached the highest AUC of 93.6% and the highest F1-score of 89.8%, outperforming FDA, PCA/SVD, PBCT, Z-score normalization, and histogram matching.

For ResNet50 it improved AUC from 65.3% on raw images to 70.3%, a 7.7% relative gain, and F1-score from 85.1% to 86.9%. The clearest single gain came from InceptionV3, where AUC rose from 71.4% to 93.3%, a 30.7% improvement, and F1-score rose from 81.8% to 91.7%. That 30.7% figure is the source of the headline number in the abstract.

The operating-point results on DS3 add a different kind of evidence. For InceptionV3, sensitivity rose from 95.2% to 96.0% while specificity improved from 20.0% to 95.2%. That specificity change is the most striking number in the paper, but it should be read carefully: it is a single operating point on one target dataset and one backbone, not a general claim about false-positive control across the study.

Gains on DS2 were smaller but more stable

On DS2, with DS1 as reference, the proposed harmonization gave the most stable improvement across AUC and F1-score for all three backbones. For YOLOv7, AUC improved from 77.3% to 79.2% and F1-score from 84.9% to 87.3%, outperforming FDA and PBCT while staying close to PCA/SVD on F1-score.

For ResNet50, the method achieved the best overall result on this target, with AUC at 77.7% and F1-score at 90.2%, higher than all compared state-of-the-art appearance-transfer methods. For InceptionV3, it also led, with AUC at 86.1% and F1-score at 89.3%.

The sensitivity and specificity results follow the same pattern. For YOLOv7, sensitivity improved from 77.9% to 81.8% while specificity held at 76.6%. For ResNet50, sensitivity rose from 86.6% to 91.3% with specificity stable at 53.3%.

The paper's own reading of these numbers is that some comparison methods improved one metric in selected cases but were not consistent across all backbones, whereas the proposed method improved or preserved both AUC and F1-score across YOLOv7, ResNet50, and InceptionV3. That consistency claim, not any single peak value, is the paper's central comparative argument.

What the AUC and F1 numbers do not say

AUC and F1-score answer different questions here. AUC reflects probability-ranking ability and is comparatively less sensitive to class imbalance, while F1-score at a threshold of 0.5 is threshold-dependent. A method can improve one and not the other, which is exactly what the paper reports for some comparison methods on some backbones.

The reported percentages also have different denominators depending on the comparison. The 30.7% figure is a relative AUC improvement for InceptionV3 on DS3, not an absolute accuracy gain and not a detection rate. The specificity change from 20.0% to 95.2% is an absolute percentage-point change at one operating point. Mixing these two kinds of numbers would misstate what the study found.

The evaluation is also image-level for classification metrics, with an image counted positive if any polyp-class detection survived non-maximum suppression. That aggregation choice means the reported sensitivity and specificity describe image-level decisions, not lesion-level detection performance.

Where the conclusion transfers and where it stops

The evidence supports a specific claim: for polyp detection in flexible colonoscopy and capsule endoscopy datasets, a fixed CIE-Lab reference-based harmonization applied as preprocessing improved cross-dataset performance without retraining, and it did so more consistently across YOLOv7, ResNet50, and InceptionV3 than the compared appearance-transfer methods.

The scope is bounded by the datasets used. The flexible track used DS1 as reference with DS2 and DS3 as targets, and the capsule track used DS4 as reference with DS5 as target. Target sets were small, ranging from 44 polyp and 35 normal images on DS5 to 127 polyp and 53 normal images on DS2, so the per-dataset numbers rest on limited image counts.

The method also assumes a suitable reference dataset is available, ideally acquired with the same device as the training set, since the reference statistics are computed once and then fixed. The paper does not test what happens when the reference is a poor match for the training domain, and it does not evaluate modalities or tasks beyond polyp detection in these colonoscopy and capsule datasets.

The paper's own interpretation is that a substantial part of the generalization gap is driven by low-level appearance variation that can be mitigated without retraining. That is a claim about how much of the gap is appearance-related in these experiments, not a claim that appearance is the only source of cross-dataset degradation.

The practical priority is consistency, not peak scores

The most decision-relevant result in this paper is not the 30.7% AUC gain on InceptionV3 or the specificity jump on DS3. It is that the proposed harmonization improved or preserved both AUC and F1-score across all three backbones on both flexible colonoscopy targets, while the compared methods improved one metric in selected cases without holding across architectures.

That distinction matters for anyone choosing a preprocessing step for cross-device endoscopic analysis. A method that peaks on one backbone and one target dataset is harder to rely on than one that behaves consistently across architectures, especially when the transformation is deterministic, requires no retraining, and needs no method-specific parameter tuning beyond selecting a reference dataset.

The practical threshold this paper suggests is therefore about reference selection and consistency checking rather than about chasing the largest single number. The reference dataset defines the target appearance space, and the reported gains are measured against that fixed choice, so the first question for a new deployment is whether a reference acquired under the training device's conditions is available, and the second is whether the harmonization holds across the backbones actually in use.

Sources used in this answer