NepSA: Breaking Ground in Nepali Targeted Aspect-Based Sentiment Analysis

TL;DR

With the surge of social media usage in Nepal, understanding the nuances of online discourse is more critical than ever. This paper introduces NepSA, the first Targeted Aspect-Based Sentiment Analysis (TABSA) dataset for Nepali. By focusing on YouTube comments in the "News & Politics" domain, the authors tackle the challenges of code-mixing, slang, and low-resource constraints, setting a benchmark F1 score of 81.60% for sentiment classification.

Problem & Motivation: The Low-Resource Struggle

While sentiment analysis for high-resource languages like English or German has moved toward identifying specific targets (e.g., "The food is great but the service is slow"), Nepali NLP has largely remained at the document or sentence level.

The authors identify three major hurdles:

1. Linguistic Complexity: Nepali is written in Devanagari but frequently appears in "Romanized" (transliterated) form or "code-mixed" (English + Nepali) on social media.
2. Evolving Vocabulary: Political discourse in Nepal has birthed neologisms like झोले (sycophant) and मण्डले (bootlicker) which traditional dictionaries and lemmatizers fail to process.
3. Ambiguity: A single comment might praise a journalist while attacking a politician; granular, aspect-based labels are required to untangle these sentiments.

Methodology: TABSA in the Wild

The authors collected 3,068 comments from popular Nepali YouTube news channels. The annotation process was two-fold:

1. The Schema

• Aspect Categories: General, Profanity, Violence, Feedback, Sarcasm, and Out-of-scope.
• Target Entities: Person (PER), Organization (ORG), Location (LOC), and Miscellaneous (MISC).

2. The Model Architecture

To provide a robust baseline, the team compared several architectures:

• Aspect Term Extraction (ATE): Treated as a sequence labeling task (NER-style). They utilized BiLSTM+CRF and a fine-tuned multilingual BERT.
• Sentiment Classification: They tested SVM, CNN, BiLSTM, and BERT. Crucially, they introduced an Embedding Concatenation strategy, where the feature vector of the sentence is fused with the specific aspect term’s vector to ground the sentiment in its context.

Figure 1: The fine-grained annotation schema showing relationships between specific aspect terms (e.g., "stupid guy") and targets.

Experiments & Key Findings

The results provide a fascinating look at the performance of modern NLP tools on "noisy" social media data:

• Concatenation is King: Across all models—BERT, BiLSTM, and CNN—concatenating aspect terms led to significant performance gains (see Table VIII). This validates the intuition that the model needs an explicit "pointer" to the aspect it is judging.
• BiLSTM vs. BERT: Surprisingly, for the sentiment polarity task, the BiLSTM (81.6% F1) slightly outperformed multilingual BERT (79.9% F1). The authors attribute this to BERT's pre-training on formal Wikipedia text, which struggles with the high volume of spelling errors and neologisms found in Nepali YouTube comments.
• Target vs. Aspect: The models performed excellently on recognizing People (88% F1) but struggled with Violence (36.2% F1). This highlights the subjective difficulty even human annotators face when defining the boundaries of violent or profane phrases.

Table VII: Detailed Performance Break-down across different aspects and tasks.

Critical Insight & Conclusion

The NepSA dataset is a major milestone for South Asian NLP. The study proves that "off-the-shelf" multilingual models like mBERT are not silver bullets for low-resource languages, especially when those languages are evolving rapidly on the internet.

Takeaway: Future researchers should focus on training embeddings on social media-specific corpora rather than formal news or Wikipedia to bridge the "formality gap." The researchers plan to expand this to POS tagging and cross-lingual LASER embeddings to further refine performance.

Limitations

• Subjectivity: Boundary detection for noun phrases used as aspects remains a bottleneck.
• Lemmatization: Traditional Nepali lemmatizers are too rigid for modern slang, necessitating manual intervention.

For developers and researchers in the low-resource space, this paper serves as a blueprint for building high-quality specialized datasets where generic datasets fall short.