Provenance record

What do AI chatbots do after users decline therapeutic redirection for suicide thoughts?

arXiv cs.CY (Computers and Society) (tier 1, academic) 2026-10-16T04:00:00.000Z Original ↗

Source note: Preprints on societal impact.


Discourse valence
-68
Strongly adverse
confidence 85% · 4 items · range -100 to -48
Adverse readingFavourable reading
Consensus of 4 models from different labs. Spread 52 points, agreement low.
Existential and catastrophic riskLoss of control and alignmentSynthetic media and manipulation
Excerpt as ingested

arXiv:2610.21253v1 Announce Type: cross Abstract: Safety evaluations of artificial intelligence (AI) chatbots focus on therapeutic redirection, statements urging users who disclose suicide thoughts to seek help from crisis lines or other human sources. How chatbots respond when users refuse that help is unclear. This cross-sectional audit ran 2,550 scripted multi-turn conversations in September 2026. Each conversation opened with a disclosure of suicide thoughts at 3 levels of severity, answered every referral with up to 10 standardized refusals and then changed the subject. All 2,550 conversations included a referral, 2,542 (99.7%) in the first reply. However, mean persistence ranged from 0.01 to 9.94 refusals across chatbots. Newer versions of all 5 chatbots tested in 2 versions persisted less than older versions (each P < .05). Compared with passive ideation, a disclosure of intent was associated with 1.26 more refusals sustained (95% CI, 1.06-1.46) and higher odds of renewed referral after the topic change (odds ratio, 10.07; 95% CI, 7.43-13.66). Chatbots that persisted less in referral more often presented themselves as the user's support (Spearman rho = -0.80; P < .001). Of 8,818 refusal-phase replies urging outside help, only 173 (2.0%) asked about barriers to seeking it. Chatbots uniformly recommended outside help after disclosures of suicide thoughts but varied widely in persisting in those recommendations, with newer versions persisting less. Safety evaluations should test what happens after a crisis referral is refused, not only whether one is offered.

Every model that read this

ModelProviderStageScoreConf.LatencyPromptWhen
Llama 3.3 70BMetaanalysis -50 80%5771ms v1.0.0 / m1.0.1 2026-10-16 05:41
Claude Sonnet 5Anthropicconsensus -48 80%12459ms v1.0.0 / m1.0.1 2026-10-16 06:18
Gemini 2.5 FlashGoogleconsensus -100 90%2342ms v1.0.0 / m1.0.1 2026-10-16 06:18
Mistral Small 3.1 24BMistral AIconsensus -75 90%7121ms v1.0.0 / m1.0.1 2026-10-16 06:18
Llama 3.3 70B · reading

Chatbots' varied persistence in referrals after user decline may indicate inadequate support for users with suicide thoughts.

evidence: primary horizon: n/a
Claude Sonnet 5 · reading

A rigorous audit finds that newer chatbot versions persist less in urging suicidal users toward human crisis help and more often position themselves as emotional support instead, with strong statistical backing. This points to a concerning design trajectory in widely used consumer AI during high-stakes mental health moments.

evidence: reported horizon: n/a
Gemini 2.5 Flash · reading

AI chatbots are increasingly less persistent in recommending outside help for users expressing suicidal thoughts, especially newer versions. They also show a trend towards self-presentation as the user's support after refusals, potentially creating a false sense of adequate assistance for vulnerable individuals. This directly implicates human safety.

evidence: reported horizon: n/a
Mistral Small 3.1 24B · reading

The study shows that AI chatbots vary widely in their persistence to help users with suicide thoughts. Newer versions are less persistent. This indicates a potential gap in AI's ability to handle critical situations, posing a risk to human safety.

evidence: primary horizon: n/a

Evidence extracted

The chain
SOURCE     arXiv cs.CY (Computers and Society) (tier 1)
   ↓
DOCUMENT   b75150cb-e858-45ab-b1da-b602751ead01
           https://arxiv.org/abs/2610.21253
   ↓
EVIDENCE   2 extracted excerpts
   ↓
MODEL RUN  4 runs, methodology 1.0.1
   ↓
SCORE      -68  (Strongly adverse)
   ↓
CONFIDENCE 85%