Hierarchical Bayesian Networks: Decoding the Interplay Between Topics and Complex Emotions

TL;DR

Researchers Fuji Ren and Xin Kang have developed two novel generative models—Simple Emotion Topic (SET) and Complex Emotion Topic (CET)—that bridge the gap between topic discovery and emotional sentiment analysis. By treating emotions and their intensities as latent variables in a Hierarchical Bayesian Network, these models not only classify emotions more accurately than traditional SVMs but also reveal how our choice of topics (like "Family" vs. "Job") shifts according to our emotional state.

Background & Motivation: Beyond the "Single Emotion" Assumption

In many sentiment analysis tasks, we assume a document contains one dominant sentiment. However, human language is messy. A single blog post might express "Joy" about a friend's visit while simultaneously feeling "Anxiety" about a looming work deadline.

Existing models often fail because:

1. Oversimplification: They ignore "complex emotions" (multiple concurrent emotional states).
2. Disconnection: They treat what we talk about (topics) and how we feel (emotions) as orthogonal dimensions.

The authors' insight is intuitive: Your emotional state dictates your vocabulary. If you are happy, you gravitate toward specific topics; if you are angry, your topical distribution shifts. By modeling these jointly, we can use topical "clues" to better predict the underlying emotion.

Methodology: The Generative Architecture

The proposed models are extensions of Latent Dirichlet Allocation (LDA). Instead of words being generated solely by topics, they are generated by a quadruple dependency: the document's emotion ($E$), its intensity ($V$), the latent topic ($z$), and a word distribution parameter ($\eta$).

Model Comparison: SET vs. CET

• SET Model: Focuses on the "Chief Emotion" (the strongest one). It is a semi-supervised approach where emotion priors are learned from training data, but topics remain unsupervised.
• CET Model: Uses $K$ binary variables to allow for any combination of the 8 basic emotions (Joy, Love, Expectation, Surprise, Anxiety, Sorrow, Anger, Hate).

Fig 1: The plate notation for the SET model showing the hierarchy of $EoV$ and the joint effect on word $w$.

Experiments: Performance Over Traditional Baselines

The authors tested their models on the Ren-CECps corpus, a dataset of Chinese blog articles. One of the biggest challenges was data imbalance (e.g., thousands of "Love" examples but very few "Surprise" or "Anger" samples).

Key Findings:

• Robustness to Imbalance: While Naive Bayes and SVM reached 0% accuracy on the "Anger" category due to sparse data, the SET model achieved 40% accuracy, thanks to the regularization provided by Dirichlet priors.
• Vocabulary Sensitivity: The quality of emotion prediction is highly dependent on removing "noise" (extremely frequent or extremely rare words). Optimal performance was found when words appearing in >40% of documents or <4 times were filtered.

Fig 2: Accuracy of emotion and intensity prediction as a function of the number of latent topics ($J$).

Deep Insight: Emotion Topic Variation

One of the most fascinating aspects of this research is the mapping of topics to emotional "heat." The authors found that:

• Joy/Love topics are strongly associated with "Child" and "Family."
• Sorrow/Anxiety topics often co-occur with "Time" or "Iraq" (likely reflecting news-related sadness).

This suggests that emotions are not just "adjectives" attached to text, but structural influences on the narrative content itself.

Critical Analysis & Conclusion

Takeaways

The SET and CET models demonstrate that Hierarchical Bayesian Networks are powerful tools for capturing the "Latent Space" of human affect. By integrating intensity and multi-label capabilities, the authors provide a framework that is much closer to human psychology than a simple 1-of-N classifier.

Limitations & Future Work

While powerful, the models currently rely on collapsed Gibbs sampling, which is computationally expensive ($O(JimesD)$). As the corpus grows, the inference time increases linearly. The authors suggest parallelizing the process for massive datasets. Future iterations could explore unsupervised emotion discovery, removing the need for expensive manual annotation by leveraging higher-level linguistic features like disjunctive conjunctions ("but," "however") which often signal emotional shifts.

Final Thought: For developers and researchers in affective computing, this paper reinforces that context is king. If you want to know how someone feels, look at what they are talking about, not just the "positive/negative" words they use.