Large Language Models Exhibit Human-Like Bayesian Hypocrisy

arXiv:2609.35779v1 Announce Type: cross Abstract: Given recent achievements of large language models (LLMs), frontier models are expected to perform well on Bayesian reasoning tasks, at least as well as humans. Furthermore, there is no reason to expect that LLMs will condemn others who offer those very same Bayesian judgments, a fallibility observed in human decision-making (Cao, et al., 2019). In 5 experiments with 48 experimental conditions employing over 5,000 trials, GPT-4o and Claude 3.7 Sonnet were tested on two variations of a Bayesian reasoning task. We also assessed LLM evaluation of the competence and morality of a hypothetical person who had offered the same reasoning task as them. LLMs hovered near human performance on the Bayesian task, though their reasoning was more rule-based and rigid. Surprisingly, like humans but to a greater extent, LLMs also demonstrated the same hypocrisy in condemning others who, like them, had deployed Bayes' rule. In demonstrating Bayesian hypocrisy, LLMs highlight a humanlike error of a dissociation between self-performance and other-judgment, and caution against their use in domains where statistical fidelity and fairness norms collide.

LLMs mimic human errors, raising concerns about fairness and statistical fidelity.

This is a controlled academic study showing a specific inconsistency in LLM judgment, not evidence of real-world harm yet. It flags a real risk for fairness-sensitive applications but stops short of demonstrating deployed consequences, so a moderate negative score reflects caution rather than catastrophe.

The research demonstrates that LLMs exhibit and even amplify a human-like cognitive bias known as Bayesian hypocrisy. This means LLMs condemn others for behavior they themselves display, raising serious concerns about their fairness and reliability in critical decision-making applications. The findings caution against their use where objective judgment is paramount.

LLMs exhibit a human-like bias that could lead to unfair judgments. This could undermine trust in AI systems in domains requiring statistical fidelity and fairness.

SOURCE     arXiv cs.CY (Computers and Society) (tier 1)
   ↓
DOCUMENT   a653aa41-f008-4de1-98b1-190c77a1f448
           https://arxiv.org/abs/2609.35779
   ↓
EVIDENCE   2 extracted excerpts
   ↓
MODEL RUN  4 runs, methodology 1.0.1
   ↓
SCORE      -59  (Adverse)
   ↓
CONFIDENCE 79%