FakeSens: Leveraging Human Intelligence to Combat the COVID-19 Infodemic

TL;DR

FakeSens is a specialized misinformation detection framework that treats human crowd workers as "social sensors." By building a Crowd Knowledge Graph (CoCKG) that fuses medical expertise with scalable non-expert observations, it identifies misleading COVID-19 claims that standard AI models usually miss. It introduces a clever reliability-aware mechanism to filter "noise" from non-expert contributors, resulting in a ~10% F1-score boost over prior state-of-the-art methods.

Background & Motivation: The Knowledge Gap

During the COVID-19 pandemic, we witnessed an "infodemic"—a surge of misinformation ranging from plausible medical errors (e.g., Vitamin C cures COVID) to wild conspiracy theories (e.g., 5G towers or vaccine microchips).

Traditional detection systems fail here for two reasons:

1. Lack of Specificity: General-purpose models (like HAN) focus on linguistic patterns but don't "understand" the relationship between emerging entities.
2. Static Knowledge: Traditional medical ontologies don't contain "conspiracy" entities like "5G" or "Bill Gates," making them useless for debunking non-medical COVID myths.

Methodology: The "Human Sensor" Architecture

FakeSens treats the problem as a Social Sensing task. Instead of just scraping the web, it asks humans (Experts and Non-Experts) to extract "triples" (Subject, Relation, Object) from reliable articles to build a dynamic knowledge base.

1. The COVID-19 Crowd Knowledge Graph (CoCKG)

The system utilizes Amazon Mechanical Turk to gather data.

• Experts: Healthcare workers who provide high-confidence medical facts.
• Non-Experts: General users who provide scale but might have "common misunderstandings" (e.g., confusing a drug's side effects).

2. Reliability-Aware Graph Adaption

How do you handle the "noisy" input from non-experts? FakeSens uses a Reliability-Aware Crowd Knowledge Adaptor (RCKA).

• It measures the semantic similarity between non-expert triples and expert-verified triples.
• If a non-expert claims a relationship that contradicts the logic of an expert triple for similar entities, the model downweights that information.

3. Claim Guided Propagation

The framework uses a Relational Graph Convolutional Network (RGCN). Crucially, the "attention" of the graph is guided by the social media claim itself. If a tweet mentions "masks," the model prioritizes mask-related nodes in the CoCKG to verify the claim.

Experimental Performance

The authors tested FakeSens against standard benchmarks (HAN, GUpdater, DETERRENT).

Key Insights from Results:

• Precision Power: FakeSens achieved a very high precision (0.8664), which is vital for misinformation detection—you don't want to "censor" true information by mistake.
• Ablation Study: Removing the Reliability Awareness module (\R) dropped the F1 score significantly, proving that simply "crowdsourcing" isn't enough; you must mathematically model worker reliability.

Figure: Performance increases as the size of the graph and the ratio of expert triples grow.

Critical Perspective: Limitations & Future

While FakeSens is a major step forward, its reliance on active crowdsourcing might be a bottleneck in a real-time "breaking news" scenario where even reputable articles aren't available yet.

Future Directions:

• Automating "Experts": Could LLMs act as the "Expert Workers" to bootstrap the graph?
• Cross-Domain Application: The RCKA logic could be applied to political fact-checking or detecting financial fraud where expertise is scarce.

Conclusion (Takeaway)

FakeSens proves that in the battle against misinformation, Human-AI Collaboration is superior to purely algorithmic approaches. By structuring human intelligence into a graph and filtering for reliability, we can create a system that evolves as quickly as the rumors it seeks to stop.