Large language models underestimate and partly misrepresent cultural variation in everyday norms

arXiv:2609.30896v1 Announce Type: new Abstract: A key aspect of culture is a society's norms about everyday behavior. How accurately do large language models (LLMs) represent cultural differences in such norms? To answer this question we used the recent Global Study of Everyday Norms (GSEN), which collected ratings of 150 scenarios in 90 societies, as the human benchmark. We prompted GPT-5 to estimate each society's average rating for every scenario, and later repeated the benchmark in three other LLMs: GPT-5.4, Claude Opus 4.6, and Gemini 3.1 Pro. Compared to GSEN estimates, all four LLMs misrepresented cultural variation in two ways. First, they greatly underestimated its magnitude, estimating differences between societies to be, on average, less than half their measured size. Second, for many scenarios the LLMs poorly identified the pattern of variation, that is, which societies judged the behavior less acceptable and which societies judged it more acceptable. The pattern of variation was identified better for scenarios that elicit concerns about vulgarity, especially scenarios involving kissing and flirting. We also found that norms in more developed societies tended to be estimated somewhat more accurately, and that prompting in local survey languages rather than English produced only a modest improvement in accuracy. Local-language prompting also reduced, but did not remove, the underestimation of between-society differences. Cultural differences in everyday norms are only weakly and unevenly represented by LLMs.

LLMs inaccurately represent cultural variation

LLMs significantly underestimate and misrepresent cultural variations in everyday norms, failing to accurately identify which societies find certain behaviors more or less acceptable. This systematic misrepresentation has strong negative implications for applications requiring cultural sensitivity, such as cross-cultural communication or ethical AI development. The findings are based on a robust comparison with a human benchmark.

The study shows that LLMs significantly misrepresent cultural variation. This could lead to misunderstandings and miscommunications in cross-cultural interactions. However, the identification of specific areas of inaccuracy and the potential for improvement through better prompting techniques offer opportunities for enhancement.

SOURCE     arXiv cs.CY (Computers and Society) (tier 1)
   ↓
DOCUMENT   b5fffa2b-c9d9-4897-9e5b-61984ad32821
           https://arxiv.org/abs/2609.30896
   ↓
EVIDENCE   2 extracted excerpts
   ↓
MODEL RUN  4 runs, methodology 1.0.1
   ↓
SCORE      -27  (Adverse)
   ↓
CONFIDENCE 84%