Teachersourcing: Can the Crowd Rival Khan Academy?

TL;DR

The bottleneck for personalized education is the lack of diverse content. This paper presents a breakthrough "teachersourcing" approach, demonstrating that ordinary workers on Mechanical Turk can produce math tutorial videos that are not only mathematically sound but also as effective as professional content from Khan Academy.

The Motivation: The Content Scarcity in Personalized Learning

For over five decades, researchers have dreamed of personalized learning systems that adapt to every student's unique struggle. However, there is a catch: Personalization requires variety. If a student doesn't understand "Explanation A," the system needs "Explanations B through Z" ready to go.

Until now, creating these resources required expensive subject matter experts. While "learnersourcing" (asking students to tag or edit content) helped, it rarely produced original, high-quality multimedia instructions. The authors of this paper ask: Can we pay ordinary people to be teachers?

Methodology: Scalable Pedagogy

The researchers targeted logarithms—a topic complex enough to require a tutorial but simple enough for an adult to recall or relearn quickly.

1. The Crowdsourcing Pipeline

Participants on Amazon Mechanical Turk (AMT) were paid $5 per video. They were given:

• Informed consent and recording releases.
• Quality guidelines (clear handwriting, vocalizing thoughts).
• Example "gold standard" videos for inspiration.

2. Validation and Pedagogy

The team focused on "worked examples," a proven pedagogical strategy where the instructor solves a problem step-by-step.

Fig 1: Examples of the diverse styles of tutorials submitted by the crowd, ranging from handwritten notes to digital whiteboards.

Experiments & Results: Crowd vs. Professional

The study wasn't just about quantity; it was about learning efficacy.

The Learning Gain Metric

Effectiveness was measured using the formula: $G_k=\frac{1}{|V(k)|}{\sum }_{i\in V(k)}(pos{t}_i-pr{e}_i)$ Where $G_k$ represents the average improvement in test scores after watching a specific video.

The Head-to-Head Comparison

The most striking result came from Experiment 3. The researchers compared the top-performing crowdsourced videos against a high-traffic tutorial from Khan Academy.

Table 1: The performance of the crowd's best efforts versus professional educational media.

Despite the Khan Academy video being significantly longer (7+ minutes vs. ~2 minutes), the crowdsourced videos held their own. This suggests that the alignment of the content to the specific problem often outweighs the "polish" of professional production.

Critical Analysis & Future Outlook

Limitations

1. Correctness Filtering: 11% of submissions contained mathematical errors. Any production system using this method would require a robust verification layer (likely a secondary crowd task or AI-based verification).
2. Breadth vs. Depth: While effective for discrete math problems, it remains to be seen if "teachersourcing" works for conceptual subjects like philosophy or complex physics.

Takeaway

This paper proves that the "average" person possesses significant latent pedagogical value. By leveraging crowdsourcing, we can move away from a "one-size-fits-all" educational model toward a diverse ecosystem of explanations. For developers of Learning Management Systems (LMS), this suggests a new path: treat your users (or the global crowd) not just as consumers, but as a scalable faculty.

Reference: Whitehill, J., & Seltzer, M. (2017). A Crowdsourcing Approach to Collecting Tutorial Videos -- Toward Personalized Learning-at-Scale. L@S 2017.