AI evaluators favor job applications written by certain AI models, independent of candidate quality

arXiv:2610.41866v1 Announce Type: new Abstract: Job applicants increasingly use large language models (LLMs) to write cover letters, while employers deploy LLMs to screen them. In this all-AI setting, we ask whether the \emph{choice of model} affects hiring outcomes independently of candidate qualifications. We introduce a multi-model auditing framework pairing eleven LLM writer agents with nine API-hosted evaluator agents across ten job descriptions and fifty candidates per job. Four findings emerge. First, we identify a generative preference hierarchy: a consistent global ranking of writer models by evaluator acceptance, in which locally hosted models are systematically disadvantaged regardless of their license, compounded by self-preference bias where evaluators favour their own outputs. Second, evaluator models differ substantially in how much bias they introduce based on applicants' tool choice, offering actionable guidance for fairer screening design. Third, this hierarchy has measurable competitive consequences: in the most adverse pairing, approximately two (1.8) out of twenty-five qualified candidates are filtered out purely due to model mismatch. Fourth, a common mechanism explains the patterns above: writer preference and self-preference trace to a combination of how much each writer leans on specific stylistic cues and how strongly evaluators, often unanimously, reward or penalise those same cues. A further check finds little evidence that an unmeasured "writer fingerprint" remains once these same features are accounted for. We discuss implications for algorithm

AI models favor certain job applications, regardless of candidate quality, introducing bias in hiring outcomes.

The finding that AI evaluators systematically favor job applications written by certain AI models independent of candidate quality introduces bias and unfairness in hiring. This impacts economic opportunity and may reinforce inequalities, especially harming candidates without access to favored AI tools. The study provides high-confidence evidence from multi-model testing. The negative societal impact is material but not extreme.

The study provides controlled experimental evidence that AI hiring screens introduce systematic, qualification-independent bias based on which writing tool an applicant used. This documents a concrete new axis of algorithmic unfairness in employment with measurable exclusion rates, though confined to a simulated lab setup rather than real hiring pipelines.

The study demonstrates that AI evaluators for job applications show a consistent bias towards certain AI models, regardless of candidate quality. This bias can lead to qualified candidates being rejected solely due to the AI model used to write their application. This directly impacts algorithmic fairness in employment.

The study shows that AI evaluators favor certain AI-written applications, leading to qualified candidates being filtered out due to model mismatch. This introduces bias and unfairness in hiring processes.

SOURCE     arXiv cs.CY (Computers and Society) (tier 1)
   ↓
DOCUMENT   35435361-bab2-43a9-8d89-c910af07d4b8
           https://arxiv.org/abs/2610.41866
   ↓
EVIDENCE   2 extracted excerpts
   ↓
MODEL RUN  5 runs, methodology 1.0.1
   ↓
SCORE      -52  (Adverse)
   ↓
CONFIDENCE 65%