How does a feedback loop actually amplify bias and hallucinations?
The feedback loop works like this: a language model generates text, a human (or automated system) provides feedback on that text, and the model is retrained to align with that feedback. If the model's initial output contains subtle biases or errors, the human feedback may reinforce those biases, and the model learns to produce even stronger versions of them in the next round. This creates a snowball effect where small biases grow into large ones. In a series of experiments with 1,401 participants, researchers found that human–AI interactions amplified biases in perceptual, emotional, and social judgments significantly more than interactions between humans did [2]. The AI's tendency to amplify biases, combined with humans' tendency to be unaware of the AI's influence, made the effect especially strong [2].
For hallucinations, the mechanism is similar: if a model generates a plausible-sounding but false statement, and a human reviewer (or automated reward system) fails to catch the error, the model learns that such fabrications are acceptable. Over successive rounds, the model may become more confident in generating false information, because the feedback loop has inadvertently rewarded it. While the papers here focus more on bias, the same principle applies to hallucinations—the loop can reinforce inaccuracies if the feedback signal is noisy or biased.
What does the research show about how much bias can grow?
The evidence shows that bias amplification is measurable and can be substantial. In knowledge base question generation, a bias-corrected reinforcement learning from human feedback (RLHF) framework improved performance by 24.5% on one dataset, 4.21% on another, and 2.16% on a third, compared to existing methods [1]. These gains came specifically from reducing sycophancy and confirmation biases in the feedback loop—meaning that without correction, those biases were dragging performance down by those amounts [1]. Another study found that RLHF can perpetuate biases present in human feedback, and that without robust mitigation mechanisms, models can become more biased over time [4].
The amplification is not just a theoretical concern. In the human-AI interaction experiments, the feedback loop caused small initial errors in judgment to escalate into much larger ones, across multiple domains [2]. The effect was significantly larger than what happens when humans interact with other humans, because AI systems tend to amplify biases more aggressively and humans perceive AI as more objective than it really is [2]. This means that even a small initial bias—like a slight preference for one demographic group—can grow into a much larger disparity after several rounds of feedback.
Why does the feedback loop make writing more generic?
The feedback loop tends to push models toward safe, conventional, and generic outputs because human feedback often rewards responses that are polite, uncontroversial, and aligned with common language conventions. A rhetorical analysis of RLHF-enhanced chatbots found that the procedure reinforces hegemonic language use and decontextualizes learning, meaning the model learns to produce text that is broadly acceptable but lacks the diversity and creativity of human writing [3]. This is because the feedback mechanism implicitly penalizes novelty or risk-taking, even when those qualities might be desirable.
The problem is compounded by the fact that RLHF is designed to maximize human approval, which often means avoiding anything that could be seen as offensive, unusual, or incorrect. As a result, models converge on a narrow range of safe, generic responses. This is similar to the mode collapse problem seen in generative adversarial networks (GANs), where the model produces only a limited variety of outputs [5]. The feedback loop can thus reduce the diversity of generated text, making it sound bland and formulaic. One study on generative relevance feedback found that varying the generation subtask (e.g., queries vs. essays) could improve diversity, but the default RLHF training tends to push in the opposite direction [6].
About These Sources
This answer is built on 6 peer-reviewed studies — published from 2023 to 2025, 4 from 2024 or later, 3 in Q1 journals, collectively cited 1,108 times — selected as the most relevant from 6 studies that passed quality screening, drawn from 47 papers retrieved from a database of over 500 million.
Sources used in this answer
Towards human-like questioning: Knowledge base question generation with bias-corrected reinforcement learning from human feedback
A bias-corrected RLHF framework for knowledge base question generation reduced sycophancy and confirmation biases, achieving relative performance gains of 24.5%, 4.21%, and 2.16% on three datasets compared to existing methods.
How human–AI feedback loops alter human perceptual, emotional and social judgements
In experiments with 1,401 participants, human–AI feedback loops amplified biases in perceptual, emotional, and social judgments significantly more than human–human interactions, because AI systems amplify biases and humans are often unaware of the AI's influence.
Ethics and Persuasion in Reinforcement Learning from Human Feedback: A Procedural Rhetorical Approach
A rhetorical analysis of RLHF-enhanced chatbots found that the procedure reinforces hegemonic language use, perpetuates biases, decontextualizes learning, and encroaches on human relationships, leading to generic and less diverse outputs.
Mitigating Bias in Reinforcement Learning from Human Feedback for Large Language Models
A comprehensive study on mitigating bias in RLHF for LLMs found that human feedback can introduce biases, and proposed mechanisms to reduce these biases, demonstrating improved alignment with ethical standards.
Generative AI
A review of generative AI noted that GAN training often suffers from non-convergence, mode collapse, and hyperparameter sensitivity, which can limit the diversity and creativity of generated content.
Generative Relevance Feedback with Large Language Models
Generative Relevance Feedback using LLMs improved retrieval effectiveness by 5-19% in MAP and 17-24% in NDCG@10 compared to pseudo-relevance feedback, showing that varying generation subtasks can improve output diversity.