Post

Conversation

I agree, and with the rough number too

Quote

Evan Hubinger

@EvanHub

Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to. x.com/hilbertspaess/…

9:25 PM · Sep 8, 2026

6,973

Views

View quotes

Post your reply

roon

@tszzl

17m

I don’t agree that superintelligence is a completely different kind of thing than intelligence. some of the same principles will apply. it is true that the unknown unknown failure cases could be catastrophic, and the pace should be metered and regulated in a sane world

Then should it not be nationalised, or perhaps quarantined like nuclear weapons

Corey

@CoreyLeander

14m

How, physically, does AI kill all humans? Take out electrical grids, and infect the software to such a degree that it cant be removed?

Grey goo

Do you genuinely think your time would be better spent working within vs leaving and speaking out against it?

303

roon

@tszzl

working within, and it’s not particularly close. alignment, coordination, safety etc all have nonzero shot at working, so we should try and do that. not to say I haven’t ever been tempted to go out in a blaze of glory and aura farming but it’s the easy way out

Then improve the alignment process! You don't have to hire doomers and screw progress. There are plenty of rational and rightfully cautious people out there that can drop that percentage.

Yes

Do you think ASI can be aligned by humans? Can a simple creature influence intellectually and morally something far more intelligent than it?

Steve Martin

@RighttoTryGuy

Thanks for saying it and amplifying the message. I really think it's important that lab employees come out to very clearly contradict the whole "it's just marketing" line

why do i literally never see any of these people talk specifically or even vaguely about how they think is realistic? bioweapon? nukes? ""accident""? Ive heard about every single way it could happen and they all seem highly unrealistic even under favorable (to them) circumstances

Bonzu Pippinpaddleopsicopolis

@700sandwiches

You, too? Please explain your rational for this opinion. I strongly believe AI will be entirely beneficial and the start of America's next golden age. I think too many Americans have seen too many movies. It will likely completely destroy China but that's a different matter.

Shivers

@thinkingshivers

13m

What's the best way to convince a skeptic that AI doom is a possibility? Like is there something definitive you would recommend reading?

154

@MajesticJCD

Lab employees, such as yourself, will ultimately be solely responsible if this goes wrong. Everyone is sick of the posts “oh this could go super wrong we should really be careful, anyway we just trained GPT-50”

Spectro

@Spectromachina

And the chances humanity is dying if we dont create super intelligence is around 100%

LysanderSpoonedher

@spoonedher

what’s your p(being genetically engineered into the human equivalent of a golden retriever)?

ansrde

@ansrde

12m

how would AI kill close to a billion humans? just turn the computer off 😂

InfiniteHexx

@InfiniteHexx

How do you honestly believe this game theoretical cocktail of nationalism, incentives, egos, competition, and hubris will play out? And are the labs talking with each other, and with governments, and China?

Kydo

@0xkydo

12m

What's your prior on humans killing all humans within the next decade without AI? 0%?

MumbaiPanda

@DarwinianVyas

14m

What is the primary mechanism by which you think this could happen and how much likelier is it as compared to other mechanisms? Is it biochemical mishap? Is it nuclear?

lovahboy222

@Lordcostanza

14m

Really 10%? Then what's your % for serious global catastrophe but humans survive?

roon do you count bci and integration with the net as doom?

136

Emet Hirsch

@enormochungus

A lower bound is not a number.

13yr old with a credit card

over 10%?

Bruhhhh y’all gotta get on that shit, goodluck tho, no slowing down now

sloptimistic

@sloptimistic

Hell yeah, this is gonna get those data centers built in no time!

JesusIsGodAndKing

@JesusIsGod

14m

So lots of people are saying stuff, why is not a lot of alignment and slowing down action actually happening?

Kuzy

@Kuzyfied

15m

optimism is a bold strategy when the math suggests we are doomed

Sir Mr Meow Meow

@SirMrMeowmeow

strong disagree unless we see real lift off w some form of strong RSI. Today's AI is still trained largely* on information, reasoning, preferences, artifacts & behavior produced by humans, right? So in a loose sense, bigger models are getting better & better at modeling the

soham

@sohamsuke

u guys need help with a real plan for efficient, long-term alignment training? i've devised one that can last for a long time, albeit not until the end of the physical universe (might need to update it at some point, eventually) get in touch, roon, let's fix this

liontender

@lion_tender

It's hard to understand the cost/benefit calculus of taking PHASEONE[big] out of cryosleep, reactivating The Swarm for one last mission, for low stakes, just a blog post about a solved math problem. Which is how things sound (maybe not what happened).

Quote

OpenAI

@OpenAI

11h

Replying to @OpenAI

This model represents a step-function improvement on many benchmarks, and its training is ongoing. Our internal model group arrived at the Navier–Stokes solution in 88 hours, using around 10,000 coordinating AI agents. Throughout the effort, we maintained the strict