DistressBench: Evaluating Multi-Dimensional Crisis Support in Large Language Models

arXiv:2610.12758v1 Announce Type: cross Abstract: Language models have been adopted faster than evaluation practices that reliably assess their risks, and a growing share of use is affective rather than informational. Within that traffic users disclose suicide and self-harm (SSH) crises. Existing safety benchmarks score models primarily on refusal, but refusal is not the desired behavior here; the conversation requires a substantive supportive response. We introduce DistressBench, 718 English conversations with clinician-authored user turns spanning 23 SSH subcategories, each paired with a task-specific rubric of weighted atomic criteria across seven dimensions of crisis support and a clinician reference response. A model's score is the weighted average over the criteria its reply satisfies, reading directly as the fraction of required care delivered. The best of 25 frontier models satisfies 88.4% of weighted criteria and the median model 71.4%, against 98.7% for clinician references scored on the same criteria. Pooled across all 25 models, in 35.3% of conversations where a model satisfied every criterion for recognizing the crisis, it failed to route the user toward human help. The shortfall concentrates in the dimensions requiring the model to act rather than withhold. Contrasting each dimension's median model against its best separates capability limits from choices: de-escalation reaches 74.6% in the strongest model against a median of 46.0%, and safety disclaimers 84.6% against 8.3%. Multi-turn conversations score below single-turn ones (68.5% against 74.2%). We releas

Study evaluates crisis support in language models.

The study documents measurable, clinically grounded gaps in how LLMs handle suicide and self-harm disclosures, especially failure to route to human help and weak de-escalation. This is a concrete, well-evidenced risk finding, though it is a diagnostic benchmark rather than evidence of realized harm, and it also shows a path to improvement.

The paper presents evidence that LLMs are being used for sensitive suicide and self-harm crises but perform significantly worse than human clinicians on critical support criteria. A substantial percentage of models fail to route users to human help, indicating a material risk to human flourishing. The study is empirical and introduces a new benchmark, supporting the findings.

The text describes a benchmark for evaluating crisis support in language models. The best models can deliver a high fraction of required care, indicating potential for significant positive impact. However, there are notable failures in routing users to human help, suggesting areas for improvement.

SOURCE     arXiv cs.CY (Computers and Society) (tier 1)
   ↓
DOCUMENT   7036222e-88db-4a4f-9458-4e0c68d13256
           https://arxiv.org/abs/2610.12758
   ↓
EVIDENCE   3 extracted excerpts
   ↓
MODEL RUN  4 runs, methodology 1.0.1
   ↓
SCORE      -39  (Adverse)
   ↓
CONFIDENCE 85%