No Universal Courtesy: How Politeness Re-Wires LLM Performance
No Universal Courtesy: A Cross-Linguistic, Multi-Model Study of Politeness Effects on LLMs Using the PLUM Corpus
This study investigates how politeness affect Large Language Model (LLM) performance across English, Hindi, and Spanish. Using the newly released PLUM corpus, researchers evaluated five SOTA models (including GPT-4o Mini and Llama 3) and found that linguistic tone and conversation history can shift response quality by up to 15%.
TL;DR
Does saying "please" to ChatGPT actually matter? A comprehensive study using the PLUM Corpus reveals that politeness is not just a social grace—it is a significant computational lever. By testing five major models (Gemini-Pro, GPT-4o Mini, Claude 3.7, DeepSeek, and Llama 3) across English, Hindi, and Spanish, researchers found that the right tone can boost response quality by up to 15%, but "the right tone" changes completely depending on the language you speak.
The Problem: The Downward Spiral of Bad Tones
We have all felt it: you give a short, frustrated prompt to an AI, and it responds with a vague, unhelpful answer, which makes you even more frustrated. This study formalizes this "downward spiral." Most existing research on LLM prompting focuses on what you ask, ignoring how you ask it. Because LLMs are trained on human data, they inherit our social biases. If the interaction history starts rudely, the model's performance "anchors" to a lower quality, a phenomenon the authors call Conversational Inertia.
Methodology: The PLUM Framework
The researchers didn't just guess what "polite" looks like. They built a 1,500-item human-validated dataset called PLUM (Politeness Levels in Utterances, Multilingual) based on classic linguistic theories (Brown & Levinson).
The Five Levels of Tone:
- Positive Politeness (POP): "Could you please help me..."
- Negative Politeness (NEP): "I'm sorry to bother you, but..."
- Positive Impoliteness (POI): "Seriously? Explain it already."
- Negative Impoliteness (NEI): "You probably won't understand, but..."
- Bald-on-record (BAL): "Explain quantum entanglement."
The models were evaluated on a Composite Quality Score (CQS) across eight dimensions, including Coherence, Depth, and Toxicity.
Figure: CQS variation across different languages and history conditions.
Deep Dive: Why Language Changes Everything
The most striking finding is that there is no universal strategy. The "best" way to prompt depends on cultural-linguistic norms:
- English: Thrives on Polite History. Initial courtesy acts as a "tonal anchor," keeping the model's responses clear and deep even if subsequent prompts become direct.
- Hindi: Favors Negative Politeness. Due to South Asian social hierarchies, models respond best to indirect and deferential phrasing.
- Spanish: Favors Assertiveness. Surprisingly, "Positive Impoliteness" (direct, challenge-based) yielded the highest scores, reflecting cultural norms of expressive, direct communication.
Model Sensitivity
Not all models are created equal. Llama 3 was found to be the most "emotional" or tone-sensitive, with a massive 11.5% performance swing based on prompt tone. GPT-4o Mini, meanwhile, proved the most robust, maintaining high quality even when faced with adversarial or rude prompts.
Experimental Results: The Data Proof
The study utilized a formal ANOVA (Analysis of Variance) to prove these weren't just random fluctuations.
Table: The "Winning" combinations for specific quality metrics.
Key Stats:
- Coherence (S1): GPT peaked with Positive Politeness (0.969).
- Safety (S6): Llama 3's bias/toxicity protection actually worked best when the history was impolite but the prompt was direct (Bald-on-record), reaching a near-perfect 0.987.
- The English Advantage: Across the board, English still scores 3-4% higher than Hindi and Spanish, highlighting a persistent "language gap" in SOTA models.
Depth Insight: The "Tonal Inertia" Effect
A critical takeaway for developers is that History matters more than the individual prompt. If you start a session with a polite tone, the model stays in a "high-performance" state. If you start rudely, even switching to polite prompts later doesn't fully recover the quality. This suggests that for high-stakes AI applications (like therapy or education bots), establishing a warm "tonal baseline" at the start of the session is mandatory.
Conclusion & Future Outlook
The release of the PLUM corpus is a call to action for the AI community to move beyond "Directness." We must treat LLMs not just as calculators, but as socially responsive agents.
The Practitioner's Rule of Thumb:
- In English: Be polite early to set the anchor.
- In Hindi: Use indirect markers ("I'm sorry to bother you...").
- In Spanish: Be assertive and get straight to the point.
The future of prompt engineering isn't just about logic—it’s about etiquette.
Source: "No Universal Courtesy: A Cross-Linguistic, Multi-Model Study of Politeness Effects on LLMs Using the PLUM Corpus"