Unsupervised Suicide Detection: Can Machines Identify Crisis Without Labels?

TL;DR

To combat the bottleneck of manual data labeling in mental health research, this study proposes an unsupervised pipeline to categorize suicidal messages. Using semantic measures like Word Mover's Distance (WMD) and clustering algorithms, it achieves up to 87% F1-score in identifying "Risk" vs "No Risk" messages in Spanish and 79% in English, offering a path toward semi-automated corpus annotation.

Background: The Annotation Bottleneck

Early intervention is key to suicide prevention, yet health professionals cannot monitor the vast scale of social media manually. While AI offers a solution, most NLP models are supervised, requiring thousands of manually labeled examples. In the sensitive domain of suicide detection, such data is scarce and expensive to produce. The authors of this paper ask: Can we identify risk patterns using the inherent semantic structure of the language itself?

Methodology: From Words to Clusters

The researchers developed a five-stage pipeline designed to transition from raw social media text to actionable risk categories.

1. Semantic Similarity: The Core Engine

Rather than simple keyword matching, the study explores three advanced metrics:

• Latent Semantic Indexing (LSI): To find "hidden" relationships between terms.
• Word Mover’s Distance (WMD): A distance metric that calculates the "travel cost" between document word clouds, using Google's Word2Vec embeddings.
• Cosine Similarity: Measuring the angle between vector representations.

2. Clustering Logic

The paper compares K-means, PAM (Partitioning Around Medoids), and Hierarchical Clustering. The goal was to see if these algorithms could naturally group "No Risk" messages away from "Immediate," "Urgent," or "Possible" risk levels.

Figure 1: The proposed pipeline from preprocessing to validation.

Experimental Insights: English vs. Spanish

The study utilized the Life Corpus, a specialized dataset containing 102 texts across both languages.

Binary Classification (Risk vs. No Risk)

The results were surprisingly robust for an unsupervised method. Using Hierarchical Average Linkage, the system achieved:

• Spanish: 87% F1-score (using WMD).
• English: 79% F1-score (using LSI).

This suggests that "Risk" messages share a high degree of semantic overlap that distinctively separates them from general social media chatter.

Figure 2: Performance distribution across various algorithm/similarity combinations. Triangle shapes represent the mean.

The Limitation: The Multi-Class Challenge

While the system succeeded at binary "Risk/No Risk" detection, it struggled significantly with Multiple Categorization (e.g., distinguishing "Urgent" from "Possible"). The Adjusted Rand Index (ARI)—a measure of how well the clusters match ground truth labels—stayed quite low (best result ~0.12).

This indicates that while the presence of risk is semantically distinct, the intensity or urgency of that risk is much more nuanced and may require supervised context or longitudinal behavioral data to solve.

Conclusion and Future Outlook

This work provides a critical "starting point" for semi-automatic annotation. By using these unsupervised clusters to find potential risk messages first, researchers can significantly reduce the workload of human annotators.

Future Directions:

• Scaling Data: The current study used a small corpus (102 texts). Larger datasets are needed to stabilize the clusters.
• Advanced Embeddings: Moving from Word2Vec to Contextual Embeddings (like BERT or RoBERTa) could better capture the subtle emotional shifts in suicidal ideation.
• Semi-Supervised Hybrid: Using the cluster outputs as "pseudo-labels" for a supervised model could further bridge the performance gap.

Note: If you or someone you know is in crisis, please reach out to local mental health services or a suicide prevention hotline immediately.