Bridging the Gap in Crowdsourcing RE: Automated User Request Classification

TL;DR

Requirements Engineering (RE) is evolving from closed-room stakeholder meetings to open crowdsourcing. However, this shift creates a "data deluge" of informal user requests. This paper presents a specialized machine learning framework using Support Vector Machines (SVM) and Active Learning to automatically categorize these requests into structured requirement types, specifically tackling the "imbalanced data" problem where critical requirements like Security are often rare.

Problem & Motivation: The Noise in the Crowd

In platforms like SourceForge or GitHub, users constantly propose new features or report issues. These are "User Requests"—informal, unstructured, and messy. To turn them into professional Software Requirement Specifications (SRS), analysts must first classify them (e.g., "Is this a performance issue or a new capability?").

Prior works often treated this as a standard text classification task. The problem?

1. Data Imbalance: 80% of requests are usually about "Capabilities," while critical "Security" or "Reliability" requests make up less than 5%.
2. Low Quality: Informal language lacks the formal structure of standard SRS documents.
3. Expertise Bottleneck: Creating labeled training data for every new software project is prohibitively expensive.

Methodology: Keywords & Heuristics

The authors' core "Insight" is that requirement types have a distinct "linguistic signature." They move beyond simple Bag-of-Words (BoW) by introducing three layers of features:

1. The Keyword Strategy

• Non-Project-Specific: Words like "encrypt," "password," or "safeguard" always imply Security, regardless of the project.
• Project-Specific: Using Word2vec, the model finds words related to the project's core domain (e.g., "database" for a password manager).

2. Heuristic Properties (HP)

Instead of just looking at what the user says, the model looks at how they say it. They defined 7 HPs:

• Rationale: Sentences containing "so that" or "because."
• Expected Behavior: Sentences using "should" or "instead of."
• Context: Sentences using "when," "if," or "while."

3. Active Learning

To solve the "labeling effort" problem, they used Active Learning. The model identifies which unlabeled requests it is most "uncertain" about and asks a human to label only those. This allows the model to reach peak accuracy with much less data.

Experiments: Proving the Value

The methodology was tested on three major open-source projects: KeePass, Mumble, and WinMerge.

Key Findings:

• SVM dominance: SVM-based classifiers significantly outperformed Naïve Bayes and k-Nearest Neighbor in this domain.
• Minority Class Boost: By using targeted keywords, the recall for rare classes like "Security" improved dramatically compared to standard TF-IDF approaches.
• Efficiency: Active Learning curves showed that the model converges to high accuracy much faster when using keyword-based features.

Critical Analysis & Conclusion

This paper serves as a bridge between traditional Natural Language Processing and modern Crowdsourcing RE. While the field is now moving toward Large Language Models (LLMs), this research highlights a fundamental truth: Structured linguistic prior knowledge (Ontologies/Keywords) is a powerful tool for handling imbalanced data.

Limitations:

• The framework assumes a request only belongs to one class. In reality, a request could be both a "Capability" and a "Security" concern.
• The project-specific keyword extraction still requires some manual validation to ensure the Word2vec output makes "intuitive sense."

Future Outlook: The combination of Heuristic Properties (structural intent) with modern Embeddings (like BERT or GPT) could represent the next frontier in making crowdsourced requirements truly actionable for software engineers.