GEOSPOT: Quantifying the "Where" in Model Generalization via Optimal Transport

TL;DR

Geospatial distribution shift is a silent killer of model performance. A model trained on houses in the USA often fails in Egypt due to cultural and architectural variance. GEOSPOT bridges this gap by using Optimal Transport to calculate a "Geospatial Distance" that combines visual/textual features with geographic coordinates. It can predict how much a model's accuracy will drop in a new country before you even collect a single image there.

Background: The Problem with Nearness

Tobler’s First Law of Geography states that "near things are more related than distant things." In ML, we've historically used this in two disconnected ways:

1. Geodesic Distance: Physical distance (kilometers). Great for intuition, but ignores that a city in Australia might look more like a city in California than one in rural Mexico.
2. Feature Distance: Comparing image embeddings. Highly accurate but requires you to already have data from the target region—defeating the purpose of "predicting" transfer success.

Methodology: The GEOSPOT Framework

The authors propose a hybrid metric that lives in a joint space $Z=XimesL$ (Features $imes$ Location).

1. The Ground Cost

The core innovation is the pointwise distance $d_Z$: $d_{\mathcal{Z}}(z,z^{\prime }{)}^p=\lambda d_{\mathcal{X}}(x,x^{\prime }{)}^p+(1-\lambda )d_{\mathcal{L}}(\ell ,{\ell }^{\prime }{)}^p$ Where $\lambda$ balances feature similarity and geographic proximity.

2. Location Encoders as Proxies

Instead of raw lat/lon, they use pretrained location encoders (SatCLIP, GeoCLIP). These encoders are "geography-aware" because they were trained to match coordinates with satellite/Flickr imagery. They effectively turn a coordinate into a semantically rich vector that describes the character of a place.

Figure 1: GEOSPOT calculates transport costs by matching distributions across both visual features and geographic context.

Experiments & Results

The authors tested GEOSPOT across benchmarks like Geo-YFCC, FMoW-Wilds, and GeoDE.

Predicting Performance Degradation

The primary metric was $\Delta$ (the drop in accuracy when moving from Source to Target). GEOSPOT distances (especially using GeoCLIP) showed much higher correlation ($|\rho |\approx 0.74$) with these drops than simple geodesic distance ($|\rho |\approx 0.42$).

Guided Data Selection

Can GEOSPOT help us pick the best "proxy" data when we can't train on the target? Using a greedy selection algorithm (Algorithm 1), the authors showed that choosing domains that minimally pooled GEOSPOT distance to the target yielded significantly better models than random selection.

Table 1: Comparison of embedding modalities. Combining ResNet50 with GeoCLIP consistently yields the highest predictive power for transferability.

Deep Insight: Applicability Maps

One of the most practical outputs of this research is the Applicability Map. By calculating the GEOSPOT distance from a source (e.g., the US) to the rest of the world, we can visualize the "safety zone" of a model.

Figure 2: Global applicability maps for models trained in the US and Brazil. Yellow indicates high transferability, while purple suggests a need for local data.

Critical Analysis & Conclusion

Takeaway

GEOSPOT proves that location is a feature, not just metadata. By leveraging Optimal Transport, we can treat whole countries as probability distributions and calculate how "heavy" the lift is to move a model from one to another.

Limitations

• $\lambda$ Hyperparameter: The balance between features and location currently requires manual tuning or heuristics.
• Encoder Bias: The performance is heavily dependent on the quality of the pretrained location encoder. If the encoder hasn't seen enough of the "Global South," the distance estimates may be skewed.

Future Work

Expanding GEOSPOT beyond classification to regression (e.g., crop yield) or segmentation (e.g., disaster mapping) will be the true test of its robustness in high-stakes geosciences.