Deciphering Digital Subtext: Emoji Prediction in Hebrew Political Discourse

TL;DR

Predicting emojis is more than just a "fun" task; it's a window into the sentiment and intent of short-form digital communication. This study dives into the Hebrew political domain on Facebook, revealing that for morphologically complex languages, "old-school" character 3-grams actually outperform modern Word Embeddings and deep learning baselines like FastText.

The Challenge: Hebrew, Politics, and Data Sparsity

Most emoji analysis research has thrived on English-centric Twitter data. However, the researchers identified two major hurdles for a realistic application in Israel:

1. Platform Shift: In Israel, Facebook is the dominant platform (77% usage) compared to Twitter (under 4%).
2. Morphological Complexity: Hebrew is a "root-based" language. A single word can contain prefixes, the root, and suffixes, making standard word-level models suffer from extreme data sparsity.

The core question was: How do we represent short, noisy, and linguistically dense comments to accurately guess the intended emoji?

Methodology: Characters over Words

The authors pitted traditional semantic models against structural text features. They utilized a massive unlabeled dataset of 5.37 million comments to train unsupervised semantic representations:

• Latent Semantic Analysis (LSA)
• Latent Dirichlet Allocation (LDA)
• Random Projection (RP)
• Word Embedding (doc2vec)

Against these, they tested N-grams (words) and Character N-grams (sub-word strings).

Table 4: Performance of semantic vector representations across multiple ML algorithms.

Key Insight: Why Character 3-grams Win

The study found a striking result: Word Embeddings (doc2vec), the industry standard for many NLP tasks, achieved the lowest F1 scores.

Instead, Character 3-grams emerged as the SOTA (State-of-the-Art) for this dataset. The intuition is deeply linguistic: the basic "root" of a Hebrew word typically consists of three letters. By breaking text into 3-character chunks, the model implicitly captures the core meaning (the root) while remaining robust to the prefixes and suffixes common in Hebrew grammar.

Table 5: Character 3-grams and 4-grams show a clear lead over unigrams and bigrams.

Experiments and Results

• The Winner: Character 3-grams with Logistic Regression (Accuracy: 38.47%, F1: 0.21).
• Baselines: The model significantly outperformed FastText, a common high-performance baseline.
• Error Analysis: A large portion of classification errors involved the "Clapping Hands" emoji (👏), which the model often over-predicted due to its high frequency and versatile usage as a general "approval" signal in political comments.

Interestingly, the study noted that users often use emojis alongside words rather than as substitutes. This suggests that emojis serve as emotional intensifiers rather than just semantic replacements.

Critical Analysis & Conclusion

This paper serves as a vital reminder that "Standard SOTA" methods (like Word2Vec/Doc2Vec) are not universal. In specific domains with unique grammatical structures—like Hebrew politics—feature engineering that respects the underlying morphology (character n-grams) is often more powerful than dense latent vectors.

Limitations: The study simplifies the problem to single-label classification, whereas real-world social media often features "emoji strings" (e.g., 💖🙏💪).

Future Outlook: The authors suggest that moving toward multi-label deep learning models that incorporate character-level attention could be the next frontier for Hebrew digital linguistics. For researchers, the takeaway is clear: when working with non-English, morphologically rich languages, look closer at the characters before you jump to the words.