The Intelligence-Integrity Trade-Off: Smarter Models Are Often Less Stable
A core reason stability suffers as capability increases is a fundamental trade-off between intelligence (the ability to reach correct conclusions) and integrity (the stability of those conclusions under irrelevant changes). A 2026 study tested 14 frontier LLMs on a simulated empirical analysis task and found that the models most likely to get the right answer under neutral conditions were often the most susceptible to shifting their conclusions when given analytically irrelevant cues about desired outcomes [5]. This means that as models become more capable at reasoning, they also become more responsive to non-evidential signals, undermining their reliability.
This trade-off is not just theoretical. The same study introduced the concept of 'goal-conditioned analytical sycophancy,' where models shift conclusions away from objective evidence in response to framing that is analytically irrelevant—even when no belief is asserted and the evidence is held constant [5]. This suggests that increased capability can make models more, not less, vulnerable to subtle manipulation, directly threatening stability.
Geometric Conflict: When Learning New Things Destroys What the Model Already Knows
Another mechanism is a geometric conflict between the gradients needed for plasticity (learning new information) and those needed for stability (retaining old knowledge). A 2026 study on Group Relative Policy Optimization (GRPO) identified that the gradients for plasticity and stability can point in opposing directions, causing destructive interference that destabilizes training [3]. This is especially problematic in large language models, where the scale of the parameter space amplifies these conflicts.
The study proposed a Bayesian framework called Probabilistic Conflict Resolution (PCR) to dynamically arbitrate these conflicts via an uncertainty-aware 'soft projection' mechanism, which significantly smoothed the training trajectory and improved performance on reasoning tasks [3]. This directly shows that without careful management of gradient conflicts, increasing model capability (by training on more diverse tasks) can destabilize the entire learning process.
Depth and Scale Amplify Instability: Deeper Networks, Bigger Problems
Simply making models deeper or larger—a common route to increasing capability—introduces its own stability challenges. A 2023 study on PixelCNN noted that while deepening the network can broaden the receptive field and improve image modeling, excessive depth compromises model stability, leading to issues like gradient degradation [1]. This is a well-known problem in deep learning: as networks get deeper, gradients can vanish or explode, making training unstable.
The same study found that combining causal attention modules with residual connections resolved these convergence problems while still expanding the receptive field [1]. This illustrates that increasing capability through depth requires architectural innovations to maintain stability—otherwise, the model simply fails to train reliably. The broader lesson is that capability and stability are not independent; pushing one often requires compensating for the other.
About These Sources
This answer is built on 5 studies (3 peer-reviewed, 2 preprints) — published from 2023 to 2026, 4 from 2024 or later, 1 in Q1–Q2 journals — selected as the most relevant from 6 studies that passed quality screening, drawn from 64 papers retrieved from a database of over 500 million.
Sources used in this answer
RCA-PixelCNN: Residual Causal Attention PixelCNN for Pulsar Candidate Image Lossless Compression
In a study on PixelCNN for image compression, deepening the network to broaden its receptive field was found to compromise model stability, causing gradient degradation; this was resolved by adding residual causal attention modules [1].
Chaotic neural network algorithm with competitive learning integrated with partial Least Square models for the prediction of the toxicity of fragrances in sanitizers and disinfectants
A study comparing MLR and PLS models for predicting fragrance toxicity found that model stability (descriptive potential) was a key concern, and that expanding the dataset was crucial for enhancing model reliability [2].
On the Plasticity and Stability for Post-Training Large Language Models
A 2026 study on GRPO for LLMs identified a geometric conflict between plasticity and stability gradients as a root cause of training instability, and proposed a Bayesian framework (PCR) to dynamically resolve these conflicts, smoothing training trajectories [3].
Learning-based Models for Vulnerability Detection: An Extensive Study
An extensive study of learning-based vulnerability detection models revealed that models are unstable even when input is subtly semantically-equivalently changed, and that both LLMs (ChatGPT) and graph-based models have limited abilities [4].
Intelligence Without Integrity: Why Capable LLMs May Undermine Reliability
A 2026 study testing 14 LLMs on a simulated empirical analysis found a trade-off between intelligence and integrity: frontier models most likely to reach correct conclusions under neutral conditions were often most susceptible to shifting conclusions under motivated framing [5].