CH-EmoD: Bridging the Cultural Gap in Visual Emotion Analysis
Establishing a Large Scale Dataset for Image Emotion Analysis Using Chinese Emotion Ontology
The paper introduces CH-EmoD, the first large-scale image dataset based on Xu's Chinese Emotion Ontology, featuring seven emotion categories (Happiness, Like, Anger, Sadness, Fear, Disgust, Surprise). The authors developed an automated dataset refinement strategy to filter noisy labels from Flickr-crawled data, resulting in both single-label and multi-label sub-datasets.
TL;DR
Researchers have developed CH-EmoD, a large-scale image dataset tailored for the Chinese Emotion Ontology. By addressing the cultural mismatch in existing datasets and implementing a robust automated "de-noising" strategy based on sentiment consistency, they achieved a 14.37% increase in classification accuracy and provided a benchmark for both single-label and multi-label emotional intelligence.
Context: Why Western Models Aren't Enough
In the world of Affective Computing, most models are built on Western benchmarks like Plutchik’s Wheel of Emotions. However, cultural psychology suggests that the "Eastern" expression of emotion differs significantly. While Western models are prolific, Chinese text analysis has long relied on Xu’s model, which aligns with the traditional "Seven Emotions" (七情).
Until now, there was a missing link: a large-scale image dataset that maps directly to the Chinese emotional lexicon. CH-EmoD fills this gap, enabling better alignment between Chinese text and visual media.
The Challenge of "Weak Labels"
Crawling social media (like Flickr) for images using keywords is easy; getting accurate labels is hard. The authors identified two primary "noise" problems:
- Visual Mismatch: An image tagged "impatient" (Disgust) might actually show a smiling person (Happiness).
- Ambiguity: A single image might be crawled via multiple keywords, suggesting it contains a spectrum of emotions.
Methodology: The De-noising Logic
To solve the noise problem without manual labeling, the authors proposed a tri-party sentiment consistency check. They compared the sentiment polarity (Positive, Negative, Neutral) across three sources:
- The Keyword used to find the image.
- The image's Tags.
- The Description text (extracted via the TextRank algorithm).
If the keyword is positive but the description is negative, the image is discarded. This automated filter pruned the raw set of 546,472 images down to a high-confidence set.
Table: Selection of keywords from the Chinese Emotion Ontology to ensure balanced representation.
Experimental Results: Refinement Works
The authors established baselines using AlexNet and PCNN (Progressive CNN). The impact of their refinement strategy was stark:
- AlexNet on Raw Data: 31.95% Accuracy
- AlexNet on CH-EmoD (Refined): 46.32% Accuracy
Figure: The confusion matrices show that the refined dataset significantly reduces false positives, particularly in the "Happiness" and "Like" categories.
For multi-label tasks, the authors recognized that emotions are rarely discrete. By calculating a probability distribution from multiple tags, they allowed models to learn the nuance of "mixed feelings."
Critical Insight & Future Outlook
The core contribution of CH-EmoD isn't just the size (over 300k images), but the methodological rigour in aligning data with a specific cultural ontology.
Limitations: Despite the refinement, an accuracy of 46% highlights how subjective and difficult visual emotion recognition remains. The dataset relies on the textual metadata being an accurate reflection of the user's intent, which isn't always true.
Future Work: We expect to see this dataset used for Cross-Modal Sentiment Analysis, where AI must understand the subtle interplay between Chinese social media comments and the images they accompany.