SLSE: Mapping the Pulse of Individuals in the Social Media Era
Sentiment Evolution in Social Network Based on Joint Pre-training Model
This paper introduces SLSE (Sentiment Knowledge Enhanced Pre-training with LSTM Sentiment Evolution), a joint model designed to track and predict individual sentiment trajectories on social media. By integrating the SKEP pre-training model with an LSTM time-series module, it achieves a SOTA accuracy of 88.0% on COVID-19 related Weibo datasets.
TL;DR
Researchers have developed SLSE, a joint neural architecture that predicts how a person's sentiment changes over time on platforms like Weibo. By combining SKEP (a sentiment-specialized pre-training model) with LSTM time-series forecasting, the model achieves a remarkable 88% accuracy in predicting future sentiment polarity, significantly outperforming general-purpose LLMs and traditional lexicon-based methods.
Problem & Motivation: Beyond the Crowd
Most sentiment analysis today is "static"—it looks at a single post and labels it. When researchers do look at time, they usually focus on the "crowd" (e.g., "Is the general public happy today?").
However, understanding the individual sentiment trajectory is crucial for understanding opinion dynamics, such as how someone moves from vaccine hesitancy to acceptance. Existing models struggle because:
- Context Loss: They use simple "Positive/Negative" labels as inputs for time forecasting, losing the rich nuance of the actual text.
- Weak Embeddings: Standard BERT or RoBERTa are trained to predict words (semantics), not feelings (sentiment), making them "blind" to subtle emotional shifts.
Methodology: The SKEP-LSTM Synergy
The authors propose a two-stage pipeline called SLSE (Sentiment-LSTM Sentiment Evolution).
1. The Sentiment Encoder (Why SKEP?)
Instead of standard BERT, the authors use SKEP. SKEP is pre-trained with a "sentiment-masking" strategy. It doesn't just learn that "The [MASK] is good"; it specifically learns to predict sentiment words, polarities, and aspect-sentiment pairs. This creates a Sentiment Context Vector that is much more potent than a standard hidden state.
2. The Temporal Engine (LSTM)
Individual reviews for a single user are ordered chronologically. These SKEP-encoded vectors flow into an LSTM. The LSTM's "memory cells" are designed to capture how earlier posts (the "history") influence the sentiment of the 8th (the "future") post.
Experiments: COVID-19 Case Studies
The model was tested on 600,000 Weibo posts regarding COVID-19 vaccines and virus traceability.
Key Breakthroughs:
- Accuracy Gap: SLSE achieved 0.880 accuracy, whereas the previous "Bu-model" (based on traditional machine learning and game theory) only reached 0.733.
- Superiority Over General LLMs: Even when compared to BERT+LSTM and RoBERTa+LSTM, SLSE consistently performed better. This proves that the "Sentiment Knowledge" injected during SKEP's pre-training is the "secret sauce" for evolution tasks.
| Model | Precision | Recall | F1-Score | Accuracy |
|---|---|---|---|---|
| Bu-model | 0.690 | 0.644 | 0.666 | 0.733 |
| BERT + LSTM | 0.821 | 0.960 | 0.885 | 0.875 |
| SLSE (SKEP + LSTM) | 0.833 | 0.950 | 0.888 | 0.880 |
Critical Analysis & Conclusion
Takeaway
The core insight of this paper is that sentiment evolution is not just about time; it's about the quality of the sentiment representation. Moving from a discrete "label" (Positive/Negative) to a continuous "sentiment vector" (from SKEP) allows the LSTM to "see" the gradual warming or cooling of a user's opinion before it officially flips polarity.
Limitations & Future Work
While highly effective, the model currently relies on a fixed window (e.g., using 7 posts to predict the 8th). In the real world, the time gap between posts varies—someone might post twice in a minute or once a month. Integrating Time-Aware LSTMs or Temporal Point Processes could be the next frontier to account for the actual "irregularity" of social media posting behavior.