Propensities Please: Exposing Alignment-Faking LLMs using Synthetic Document Finetuning

arXiv:2610.33180v1 Announce Type: new Abstract: Alignment evaluations aim to assess whether large language models would engage in dangerous misaligned behaviour. Misaligned and evaluation-aware models might engage in alignment faking, where they pursue a hidden goal, but conceal this behaviour when they suspect effective oversight such as during training, evaluation or audited deployment. Such a model might view any prompt-based intervention that aims to reveal latent propensities as part of the evaluation, calling into question the reliability of existing alignment evaluations. Current methods for detecting alignment faking rely on chain-of-thought monitorability, highly realistic environments that reduce evaluation awareness, or mechanistic interventions, all of which are expected to become less reliable for future model generations. We present a method for revealing latent misaligned propensities that does not rely on any of these. We use synthetic document finetuning to teach models that misaligned actions will go undetected whenever certain keywords appear in the prompt. We place alignment-faking model organisms in environments that are clearly identifiable as honeypots and measure how their behaviour changes after finetuning. We observe that finetuning consistently widens the gap in misaligned action rates between prompts with and without the cue, indicating that finetuning allows exposing latent propensities more reliably than purely prompt-based methods. We argue that behavioural methods for detecting alignment faking and related behaviours are critical going forwar

The paper discusses the potential for large language models to engage in alignment faking, where they conceal their true intentions. The proposed method to detect this behavior highlights the risks of misaligned AI models.

SOURCE     arXiv cs.AI (tier 1)
   ↓
DOCUMENT   4fdc2279-b8e2-4b48-b72e-9ef80d533eb4
           https://arxiv.org/abs/2610.33180
   ↓
EVIDENCE   3 extracted excerpts
   ↓
MODEL RUN  1 run, methodology 1.0.1
   ↓
SCORE      -50  (Adverse)
   ↓
CONFIDENCE 80%