Provenance record

Anthropic researcher quits with a warning: Self-improving AI could "kill us all"

Ars Technica: AI (tier 1, news) 2026-09-09T16:59:40.000Z Original ↗

Discourse valence

-70

Strongly adverse

confidence 73% · 4 items · range -70 to -70

-100

-50

+50

+100

-70

Adverse readingFavourable reading

Consensus of 3 models from different labs. Spread 0 points, agreement high.

AGI and timelines Existential and catastrophic risk Loss of control and alignment

Excerpt as ingested

When a prominent researcher quits a job at a frontier AI lab these days, it's often to pursue a new startup or protest a new business model . But AI researcher Jacob Coxon is using his departure from Anthropic to publicly warn that frontier AI companies are "gambling with our lives" with systems that they "earnestly believe... could kill us all by the end of the decade." In a social media thread Tuesday night , Coxon said that this existential risk is inherent not so much in today's models but more in the impending prospect of "self-improving superintelligence" creating "superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources." Others working on these models have either not "internalized the civilizational stakes" or believe that they need to "speedrun" the race to superintelligence to prevent an irresponsible party from getting there first, he wrote. Lest you think this is just one departing researcher expressing an unpopular opinion, Anthropic Alignment Science lead Evan Hubinger piped in on social media to say that "Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade." Read full article Comments

Every model that read this

Model	Provider	Stage	Score	Conf.	Latency	Prompt	When
Llama 3.3 70B	Meta	analysis	-80	80%	3838ms	v1.0.0 / m1.0.0	2026-09-10 09:01
Llama 3.3 70B	Meta	consensus	-70	80%	4023ms	v1.0.0 / m1.0.1	2026-09-10 09:37
Mistral Small 3.1 24B	Mistral AI	consensus	-70	80%	9563ms	v1.0.0 / m1.0.1	2026-09-10 09:37
gpt-oss 120B	OpenAI	consensus	-70	60%	2943ms	v1.0.0 / m1.0.1	2026-09-10 09:37

Llama 3.3 70B · reading

Respected researcher warns of existential risk

evidence: reported horizon: n/a

Llama 3.3 70B · reading

Warnings from AI researchers about existential risk

evidence: speculative horizon: n/a

Mistral Small 3.1 24B · reading

The article presents a strong warning about the potential existential risk of self-improving AI. The claims are made by prominent researchers, indicating a high level of concern within the field. The speculative nature of the claims is acknowledged, but the severity of the potential outcomes is emphasized.

evidence: speculative horizon: n/a

gpt-oss 120B · reading

The article presents a high‑impact claim that AI could cause human extinction, which is strongly negative for humanity. However, the claim rests on personal opinions and no empirical evidence, so the negative rating is tempered by its speculative nature.

evidence: speculative horizon: n/a

Evidence extracted

Anthropic researcher believes self-improving AI could kill all humans by the end of the decade
Anthropic Alignment Science lead agrees AI could kill all humans with >10% probability within the next decade

The chain

SOURCE     Ars Technica: AI (tier 1)
   ↓
DOCUMENT   5966c46a-c5df-41a6-a8b1-8a2761074665
           https://arstechnica.com/ai/2026/09/anthropic-researcher-quits-with-a-warning-self-improving-ai-could-kill-us-all
   ↓
EVIDENCE   2 extracted excerpts
   ↓
MODEL RUN  4 runs, methodology 1.0.1
   ↓
SCORE      -70  (Strongly adverse)
   ↓
CONFIDENCE 73%