When Who You Are Can Change the Code You Get: A Study of Persona-Induced Bias in LLM Code Generation

arXiv:2609.22102v1 Announce Type: cross Abstract: Large Language Models (LLMs) are widely used as programming assistants, yet it remains unclear whether and how user's demographic information impacts the technical quality of generated code. We conduct a large-scale empirical study of persona-induced bias in LLM-based code generation, focusing a proprietary model (Gemini 2.5 Pro) and an open-weight model (GPT-OSS-120B). Using 18 demographic personas spanning nationality, gender, and experience level, we compare persona-induced prompts against a neutral baseline. Across 35,000+ generated programs, we analyze demographic marker leakage in reasoning and responses, as well as differences in functional correctness, maintainability, code style, and security. Our results show that demographic cues are frequently reflected in LLM reasoning and outputs. Demographic markers appear in up to 65% of responses and 70% of reasoning traces, despite being semantically irrelevant to the tasks. On LiveCodeBench, persona prompting were associated with lower correctness scores of the Gemini model by an average of 1.54 percentage points, with one persona exhibiting a decrease of 3.6% (odds ratio = 0.51). In contrast, the accuracy of the GPT-OSS model improved by 3.4 - 5.7% across all personas (odds ratios = 1.8 - 3.0). Maintainability and code style metrics show statistically significant but negligible effect sizes (all Cliff's {\delta} < 0.15), and security vulnerabilities exhibit no systematic persona-specific patterns. Overall, our results show that the presence of demographic information abou

Study finds bias in LLM code generation

The study finds that LLM code generation quality varies with user's demographic information, indicating persona-induced bias that affects technical outputs. This poses risks of unfairness and discrimination in software development assisted by AI, though impacts are moderate and no security vulnerabilities were detected. The evidence is based on extensive empirical data across two models.

This is a well-evidenced empirical study showing measurable but modest bias effects tied to user demographics in code generation, a widely used AI application. The magnitude is small (low single-digit percentage points) but the implication of systematic, inconsistent bias in a high-stakes technical domain is materially concerning rather than extreme.

The study provides empirical evidence that LLMs reflect user demographic cues in code generation, despite their irrelevance. One model showed decreased correctness for some personas, highlighting a measurable risk in AI-assisted software development. This directly impacts the quality of technical output based on non-technical user attributes.

The study shows that demographic information can bias LLM-generated code, affecting correctness and potentially introducing security risks. This bias can lead to unfair treatment and reduced reliability in software development.

SOURCE     arXiv cs.CY (Computers and Society) (tier 1)
   ↓
DOCUMENT   6b01f269-f3fc-4a2f-950b-4bb3dd957ffa
           https://arxiv.org/abs/2609.22102
   ↓
EVIDENCE   2 extracted excerpts
   ↓
MODEL RUN  5 runs, methodology 1.0.1
   ↓
SCORE      -65  (Strongly adverse)
   ↓
CONFIDENCE 83%