Reddit Reputation: Decoding User Trust via Community Echoes
Learning User Reputation on Reddit
The paper, "Learning User Reputation on Reddit," introduces a methodology to predict the qualities of a discussion-starting ("Parent") comment by analyzing the linguistic and numerical features of the subsequent community reactions ("Descendants"). Using a dataset of 6.8 million Reddit comments, the authors trained Random Forest and Deep Neural Network models to establish a proxy-based reputation system for detecting anti-social behavior.
TL;DR
Can we judge a person not by what they say, but by how the room reacts? This paper presents a novel approach to User Reputation Modeling on Reddit. Instead of just analyzing a single comment for toxicity, the authors look at the "Discussion Tree" it generates. By training models on how the community responds, they can predict with up to 79% accuracy whether the original post was problematic, even without looking at the post itself.
The Problem: The "Cat-and-Mouse" Game of Moderation
Traditional moderation relies on keyword filtering or NLP classifiers (like Perspective API) to flag hate speech or profanity. However, human communication is nuanced. Sophisticated anti-social behavior—sarcasm, dog-whistling, or context-heavy insults—often bypasses direct detection.
The authors argue that the impact of a message is its most defining characteristic. Anti-social behavior is, by definition, disruptive to others. Therefore, the "community response" contains a wealth of latent data that "mirrors" the quality of the instigating agent.
Methodology: Mining the Discussion Tree
The researchers analyzed nearly 7 million comments from popular subreddits like /r/politics and /r/AskReddit. They structured the data into Parents (original posts) and Descendants (the replies).
1. Feature Engineering
They didn't just look at word counts. They extracted 154 features, including:
- Linguistic Signals: Usage of 1st person ("I", "Me" - reflective/pro-social) vs. 2nd person ("You" - often accusatory).
- Sentiment & Subjectivity: Using TextBlob and profanity-checkers on the replies.
- The Disagreement Index: A custom metric calculating the ratio of downvoted to upvoted comments in a thread. High ratios indicate "mutual negativity" or a "zipper" effect where users are attacking each other.
2. Model Architecture
The authors used Random Forests (RF) for explainability (understanding which features matter most) and Deep Neural Networks (DNN) for raw predictive power.
Figure 1: The structure of a "Discussion," showing how features from Children and Descendants are used as proxies for the Parent.
Experiments and Key Findings
The models were tasked with predicting four labels for the Parent comment: Score, Sentiment, Hate Speech, and Profanity.
| Task | DNN Accuracy | Key Predictor |
|---|---|---|
| Score Prediction | 0.79 | desc_score_disag (Disagreement Index) |
| Hate Speech | 0.70 | desc_avg_hate_count (Replies often quote or mirror the hate) |
| Profanity | 0.63 | desc_avg_off_conf (Offensive replies) |
Performance Insights
As shown in the table below, the DNN consistently outperformed the Random Forest, suggesting complex non-linear relationships between a post and its subsequent discussion.
Critical Insight: The most effective feature for predicting a post's score was the Score Disagreement in its replies. If the people replying to you are all fighting and downvoting each other, there is a high statistical probability that you started the fire.
Deep Insight: The Proxy Principle
The most fascinating takeaway is the Detection by Proxy. The model found that the top predictors for profanity in a parent comment were actually profanity and offensive content in the discussion. This confirms a psychological reality: toxicity is contagious. By monitoring the "fever" (the community's angry reaction), the system can identify the "pathogen" (the original user) even if the pathogen is wearing a mask of polite language.
Conclusion & Future Work
While an accuracy of 70-79% isn't yet enough for fully autonomous "judge, jury, and executioner" moderation, it provides a powerful tool for Site Moderators.
The Roadmap Ahead:
- Scaling: Moving beyond one month of data to capture long-term user reputation.
- Beyond Text: Integrating graph-theoretical properties—how does reputation spread through "signed" networks where users follow or block each other?
- Proactive Moderation: Using these reputation scores as weights in recommendation algorithms, effectively "shadow-demoting" users who consistently trigger toxic community cycles.
This research moves us closer to a "Web of Trust" where a user's reputation is a living reflection of their impact on the digital tribe.