ChronoCulture: Evaluating Cultural Awareness Across Time in Large Language Models

arXiv:2610.20472v1 Announce Type: cross Abstract: Cultural attitudes and social norms evolve over time, yet most cultural benchmarks for large language models (LLMs) treat culture as a static attribute of a country or region. We introduce ChronoCulture, a benchmark of 10,360 temporal pairs built from longitudinal survey data across 72 countries. Each pair contains the same survey question asked in the same country in an older and a newer year, with full population response distributions as ground truth. Across six LLMs, we find that models can often predict a country's attitudes at one point in time but do not reliably capture how those attitudes change. Their predictions for different years tend to stay close together, failing to capture much of the observed change. Specifying the intended year in the prompt also offers little overall improvement in alignment with survey responses. These findings suggest that models often present a largely static picture of cultural attitudes, highlighting the need to evaluate cultural awareness across time.

LLMs fail to capture changes in cultural attitudes over time, presenting a static picture.

This is a methodological benchmark paper documenting a specific limitation in LLMs, not a claim about broad societal transformation. It has modest implications for applications relying on LLMs for social or cultural analysis, but is primarily a diagnostic contribution to AI evaluation research.

The research introduces a benchmark to evaluate how large language models (LLMs) understand evolving cultural norms. It finds that current LLMs struggle to capture temporal changes in cultural attitudes, often presenting a static view. This finding identifies a technical limitation without suggesting a clear positive or negative societal implication for human flourishing.

The study highlights a significant limitation in current LLMs, opening the door to improvements. Better cultural awareness could lead to more effective and respectful AI interactions.

SOURCE     arXiv cs.CY (Computers and Society) (tier 1)
   ↓
DOCUMENT   447a4bf3-12df-482e-a881-0d5aa166f72b
           https://arxiv.org/abs/2610.20472
   ↓
EVIDENCE   3 extracted excerpts
   ↓
MODEL RUN  4 runs, methodology 1.0.1
   ↓
SCORE      -46  (Adverse)
   ↓
CONFIDENCE 81%