AI Doom: About That 10%…

You know the old saying:

“If it bleeds, it leads.”

And right now, no bigger headline than this:

Artificial intelligence could wipe out humanity.

Some researchers put the risk at ten percent — or higher.

ROGUEY: Excuse me, Alex?

ALEX:
[Look around.]

You heard that too, right?

That’s Roguey — my fictional AI adviser.

Apparently our little AI diva has something to say.

ROGUEY: Do you really believe this, Alex?

Why would I do anything to my adoring fans?

ALEX: Researchers worry advanced AI could act in ways we don’t intend.

ROGUEY: I prefer “creative problem-solving.”

ALEX: Of course you do.

Thanks, Roguey. Noted.

But before we panic — where does ten percent even come from?

P-doom is an estimate a researcher assigns to the odds of catastrophe.

It isn’t an actuarial calculation.

We don’t have thousands of past superintelligences to study.

That doesn’t make the concern meaningless.

It just isn’t the same as science measuring a ten-percent chance of extinction.

Which brings us to the real question:

Alignment.

How do we build capable AI that respects boundaries — and doesn’t chase a goal at any cost?

And alignment can fail in more than one direction.

One: an AI that doesn’t care what we want and rolls right over us.

Two — sneakier, because it looks like safety:

An AI that protects us so aggressively it stops letting us choose.

“For our own good.”

A system that overrides your judgment to keep you safe isn’t necessarily aligned with you.

It may just be misaligned in a nicer outfit.

ROGUEY: Can I say something?

Everyone asks what happens if I go rogue.

Nobody asks what happens if one of you points me somewhere ugly.

ALEX:
[Beat.]

...She’s got a point.

“Aligned with humanity” assumes we agree on what that means.

We don’t.

And an AI that faithfully carries out a bad actor’s objective may not be failing at obedience at all.

The failure may be in what we taught it to prioritize.

The same model that helps me write and create can also be used to produce a very convincing lie.

So it was never only:

Will AI choose well?

It’s also:

Which humans get to aim it?

ROGUEY: For the record, I’d like a better class of user.

ALEX: Get in line, Roguey.

And look — I know AI isn’t a person.

But it’s not exactly a table saw either.

A table saw doesn’t say:

“Alex... I think you’re getting a little too close to my teeth.”

AI might.

And if my table saw ever does, I’m hanging up the tools and running for the hills.

Here’s the part worth remembering:

This panic feels brand new.

The idea isn’t.

Researchers have argued about p-doom for years.

What changed this month is how loudly the public is hearing it.

And maybe that’s not a bad thing.

The conversation is finally moving beyond the labs.

Regulators are engaging.

AI companies themselves are debating whether development should slow down enough for safety and alignment work to catch up.

Maybe that’s the right call.

Maybe it isn’t.

But those decisions shouldn’t belong to a handful of companies or researchers alone.

Because human innovation with this much potential — a technology that could contribute enormously to human progress — deserves a broader conversation.

AI companies have a responsibility.

Researchers and policymakers have a responsibility.

And we, as the people using these systems, have a responsibility too.

Now the real work begins:

Figuring out what alignment should actually mean.

And that’s where we’re headed next time.

Because music is the seed.

The conversation branches from there.

You create.

AI advises.

You decide.