Enhancing Text Similarity: The Power of Weighted Word Vectors and Correlation Metrics

TL;DR

Calculating semantic similarity between texts is a foundational challenge in NLP. This paper demonstrates that we don't always need complex, computationally heavy models to achieve better results. By applying Smoothed Inverse Frequency (SIF) weighting to word vectors and selecting the Pearson Correlation Coefficient as the similarity metric, the authors achieved an F1-score of 16.30—outperforming the well-known Word Mover's Distance (WMD).

Problem & Motivation: Beyond Simple Averaging

Why is measuring text similarity so difficult? If we simply average word vectors (Word2Vec), high-frequency words that carry little semantic meaning (like "the", "is", "at") dominate the resulting sentence vector.

While Word Mover's Distance (WMD) was designed to solve this by measuring the minimum "travel distance" between word clouds, it is computationally intensive and, as this paper shows, occasionally less accurate than a well-weighted vector aggregation approach. The authors' insight is to refine how we weight individual words before they are collapsed into a final text representation.

Methodology: The SIF Weighting Framework

The core of the proposed method is a weighted summation of word vectors. Instead of treating all words equally, the importance of a word $w$ in a sentence $s$ is moderated by its distance to the language distribution $L$.

The Formal Weighting Mechanism

The sentence vector $x_s$ is calculated as:

Where $\alpha$ acts as a smoothing parameter and $dis(w,L)$ represents the distance of the word from the common language model. This ensures that unique, informative words contribute more to the final vector than "noise" words.

Comparing Distance Metrics

The authors don't stop at weighting; they explore the geometry of the vector space by comparing:

• Cosine Similarity: Measures the angle between vectors.
• Pearson Correlation: Measures the linear relationship between vector components.
• Manhattan & Euclidean Distances: Traditional geometric distances.

Experiments and Significant Results

The researchers conducted a comprehensive ablation study comparing different weighting schemes (TF-IDF vs. SIF) and different similarity measurements.

Performance Comparison

The SIF-weighted word vector combined with Pearson Correlation significantly outperformed all other configurations:

Interestingly, the study found that Pearson Correlation consistently provided a slight edge over Cosine Similarity (16.30 vs 16.02), whereas distance-based metrics like Euclidean and Manhattan performed poorly (5.92), suggesting that the "magnitude" of vectors in this space is less informative than the "direction" and "correlation" of their features.

Critical Insight & Conclusion

The success of this method reaffirms a classic NLP principle: feature weighting often matters more than model complexity. By intelligently de-emphasizing frequent words through the SIF framework and using correlation-based similarity, we can achieve high-quality text matching without the metabolic cost of heavy alignment algorithms.

Limitations & Future Work

While the F1-score of 16.30 is a leap over the 10.71 of WMD, text similarity remains a challenging field. The current approach relies on pre-trained word vectors which may not capture polysemy (words with multiple meanings) as well as modern Transformer-based contextual embeddings (like BERT). A natural next step would be integrating this weighting logic with dynamic, context-aware embeddings.