Beyond Simple Metrics: Mastering User Reputation through Collective Intelligence

TL;DR

In the vast ocean of Web 2.0 content, how do we distinguish a "subject matter expert" from a "prolific spammer"? This paper proposes a modified PageRank algorithm that calculates User Reputation (UR) by analyzing the social dynamics of Q&A platforms. By blending n-gram similarity (content relevance) with collective intelligence (user selections and recommendations), it provides a sophisticated way to rank contributors and surface high-quality content even before it goes viral.

Background: The Limits of Metadata

Most social platforms rely on "shallow" features to rank answers: the number of likes, the length of the text, or the number of views. However, these are easily gamed. A user can write a long, irrelevant answer or use bots to inflate view counts. The authors argue that true content value is better estimated by the reputation of the author, which is an emergent property of their social interactions and the community's collective validation.

Methodology: A Weighted Social Graph

The core innovation lies in treating the Q&A ecosystem as a directed graph where nodes are users and questions. Unlike the original PageRank, which treats all links as equal, this method assigns specific weights based on the quality of the interaction.

1. Semantic Co-occurrence (n-grams)

To ensure an answer is actually relevant to the question, the authors use 2-gram similarity. If an answer shares significant terminology with the question, it receives a higher weight.

$s(X,Y)=\frac{|X\cap Y|}{|X|+|Y|}$

2. The Modified PageRank Formula

The reputation ($UR$) of user $p_i$ is derived from the questions ($q_i$) they answer. The link weight $f(q_i)$ distinguishes between:

• Selected Best Answers: Weight 0.8 (High trust)
• Unselected Answers: Weight 0.2 (Low trust, but still a contribution)

Figure 1: Social network structure between questions and answering users.

Experiments & Evolution

The authors tested their model on the 'GisikiN' portal dataset. The experiment evolved through three stages:

1. Selection-only Ranking: Identified active users but failed to penalize those who provided many mediocre, unselected answers (User U6 was ranked 6th).
2. Adding Similarity: Using n-grams helped refine results, boosting relevant experts.
3. Integrating Recommendations: By adding a "recommended ratio" $r(q_i)$, the model finally filtered out high-volume/low-quality users entirely.

Figure 2: Final rankings after incorporating recommendation ratios and similarity.

Critical Insight: Why This Works

This approach effectively turns a "Content Ranking" problem into a "Network Trust" problem.

• Inductive Bias: The model assumes that "Good answers come from good users."
• Resilience: By assigning a very low weight (0.1) to self-answers, it prevents users from inflating their own reputation through "alt" accounts—a common problem in modern reputation systems.

Conclusion & Future Outlook

This paper provides a robust foundation for building trust-based search engines. While 2-grams are a bit "old school" by today’s LLM standards, the logic of weighted social links remains highly relevant for modern DAO governance and decentralized social protocols (like Lens or Farcaster).

The next step for this research would be replacing n-gram similarity with transformer-based embeddings (like BERT or GPT-4) to better understand the intent beyond mere keyword matching.

Disclaimer: This analysis is based on the academic paper "User Reputation Evaluation Using Co-occurrence Feature and Collective Intelligence".