(2026-07-11) ZviM Introduction For, And Reactions To, Plan A

Zvi Mowshowitz: Introduction for and Reactions to Plan A. The folks who brought you AI 2027, a so far remarkably accurate set of predictions despite those predictions having seemed freaky to many at the time, now bring you their positive vision that involves more freaky predictions: Plan A.

These guys have rather strong prediction track records

There is also an unofficial visual novel version, for minds very different from my own who would want that. https://ai-2040.fun/

To be clear up front: I am not endorsing Plan A.
There is a lot more work to do and a lot of potential problems and downsides to grapple with.
I do think we should do that work, and give it, and its details, serious consideration.

The bulk of this post is engaging with various objections, in progressively more detail. If you only need the highlights, you can safely stop after Thus Selective Optimism.

Table of Contents

• Introducing Plan A.
• You Only Get Five Words.
• Proactive Response To Objections.
• Initial Introductions and Endorsements.
• A Positive Vision.
• Alternative Plans.
• Plan S for Shutdown.
• Something (Unexpectedly Good) Ever Happens.
• Thus Selective Optimism.
• Quickly, There’s No Time.
• Race Conditions.
• This Is A Lot Of Diffusion And Economic Growth.
• Living In China.
• Planning For Shifting Overton Windows Is Essential.
• The Standard Handwave.
• Some Equate Any Controls Over Compute To Authoritarian Dystopia And Those Same People Mostly Think Superintelligence Won’t Happen.
• Vitalik Buterin Is Right, The Crux Is Future AI Capability Levels.
• The Authoritarian Objection.
• Concepts Of A Plan.
• The Kitchen Sink.
• Selective Claims Of Authoritarianism.
• You Either Can Steer The Future Or You Cannot.
• Cooperative Alignment.

You Only Get Five Words

There is clear agreement on which sentences survive and which do not.
The top 5 things people will discuss will largely be, in descending order of focus:

• We should slow down AI development.
• To do that, we should make a deal with China.
• We should monitor the world’s major sources of compute.
• We should use mutually assured compute destruction (a version of MAIM).
• Things will still happen super fast and feel like it, e.g. ASI by 2040, with vast economic growth before this.

The strongest and loudest objection, and in some ways the best one, is some form of:
Plan A Detractors: Superintelligence (ASI) is not coming any time soon. The threat is not real, so we shouldn’t be paying high costs to deal with it. There is no reason to slow down that which is already slow enough on its own.

I appreciated Timothy Lee’s reaction even more:
Timothy B. Lee: I struggle with what to say about the new AI 2040: Plan A website. It all seems so implausible to me that I'm not sure where to start. There's an epistemic chasm between those who think superintelligence implies near-omnipotence and those (like me) who don't.

so far I’ve been unable to find a compelling argument to convince such folks that for practical purposes yes sufficiently advanced AIs could and would do all the things.

I think the premise in the Detractors argument, as stated above, is wrong. I think superintelligence is likely to be coming soon, as do the labs. Many do not agree. If you are one of those who do not agree, then you should absolutely not want to implement Plan A, or anything like Plan A, and you should say so plainly.

Top 10 criticisms other are, translated into my language (not intended to pass ITTs):

• America would never do it. You don’t understand America (or the government).
• China would never do it. You don’t understand China (or its government).
• This scenario doesn’t understand that this is a race. Or, this scenario places too much emphasis on the framing that this is now being seen and treated as a race.
• This is still too fast, or this is far too slow. Unacceptable.
• This is unnecessary, market can handle alignment, it is all easy, stop worrying. You warned things might eventually be not fine but so far everything is fine
• (5 more)

A lot of these being symmetrical is a sign that the scenario is doing something right.

My basic responses to these objections:

• Not with that attitude. If true, get cracking on figuring out an alternative.
• Not with that attitude. If true, get cracking on figuring out an alternative.
• I think this is roughly the right amount of talking like this is a race.
• Slower would be better if possible, it’s a question of what is achievable.
• If you think this, then you should oppose things like Plan A, but you’re wrong.
• (5 more)

A scenario without any unlikely events is itself even more unlikely. And in AI, the one thing I know for sure is that a lot of big somethings are going to happen. Results literally range from maximally bad to maximally good, and ~zero movement is not that likely.

Extended versions of most of these are included later in the post.

Proactive Response To Objections

There is a long tail of other objections as well. Even after listing the top 10 I found that a large percentage of responses to my open thread were not covered.
AI 2040, like AI 2027, tries to proactively answer many of the concerns and objections people have, both common and rare, including via supplementary material.

Initial Introductions and Endorsements

Scott Alexander, one of those who worked on it, writes an introduction and justification here.

Whereas here are some shorter pitches for why you should read it and it matters:

Romeo Dean: At some point soon, humanity will be forced to reckon with the creation of AIs that are smarter than humans in every way. This is a terrifying prospect. Plan A is our vision of how we can make this go well if we rise to the occasion

Ryan Greenblatt: Plan A seems like a good plan for handling powerful AI, or at least the best plan anyone’s written up. Many choices initially seem crazy, but are actually pretty carefully considered. Plan A isn’t likely to happen, but pushing for something like this seems worthwhile.
A nice property of Plan A is that it’s reasonably robust to partial or lower-quality implementation. So nearby proposals still seem good (and worse-done versions can bootstrap into better ones). The core—verification and (very strong) transparency—suffices to go pretty far.

Jonny Miller: Someone should fund turning this into a Netflix-quality production to give normies a heads-up of what is coming down the pike

A Positive Vision

They call Plan A a positive vision.
One could also call it an optimistic scenario, as per Richard Ngo. (see below)

One strong criticism of AI 2027 was that it laid out two scenarios, but did not tell us what we should do or offer us a practical path forward. In the default scenario, which lays out what the authors expect to happen, we all die.
In AI 2027’s alternative scenario we do a hard reset at a crucial moment, which is less a strategy or plan or positive vision and more like a hail mary pass, where the authors make things turn out well to show that it is in theory possible.

Plan A updates to start from our present situation. Rather than being primarily a prediction, it lays out a positive vision of a possible future where we coordinate (including using various enforcement and verification mechanisms) to slow down the development of superintelligence (ASI), and give it the best chance to go well. Ultimately in 2040 the torch is passed to the AIs, the singularity proceeds in earnest, and hopefully we got it right on the first try and things work out.

The proposed implementation of Plan A is that America and China reach a mutually beneficial deal to slow down AI development and share research information. Joint control is established over existing and new chip supply, with common knowledge of the location of existing concentrations of chips, and universal auditing of data centers

During the transition, America experiences explosive economic growth, and uses some of that to pay out a rapidly increasing Citizen’s Dividend, as an extremely generous form of UBI, and also a small fraction is devoted to various forms of defensive acceleration

If things going vertical in 2040 still sounds super fast, that is because in the authors’ current baseline or default scenario this happens in 2030.

A key focus of Plan A is total research transparency. Daniel Kokotajlo made this diagram to illustrate why he cares so much, with further explanation at the link.

Alternative Plans

They briefly also describe the scenarios where we instead choose Plans B, C, D or S:

• Plan B: Fight China, as in try to sabotage other efforts to buy a time buffer).
• Plan C: Burn the Lead, roughly the AI 2027 scenario, with a bit more awareness.
• Plan D: Race to ASI, the path we were on before Mythos and mostly are still on.
• Plan S: Shut It All Down, which is what it sounds like. They are sympathetic but don’t recommend because it would be unstable and hard to get buy-in for

Roughly speaking, they predict the presidential election in 2028 is all about AI because by that point it is obvious AI is the Main Thing, with candidates throwing around bold agendas. Then in 2029 one gets implemented, which they are hoping is Plan A.

Plan S for Shutdown

Nate Soares (MIRI): The AI futures folk and I agree on quite a lot. This is the best concrete vision of a positive future I’ve ever seen spelled out, bar none.
I doubt their Plan A would work as written, but it’d have a chance of producing evidence that convinced world leaders to switch to Plan S, which I think has a shot

Something (Unexpectedly Good) Ever Happens

Plan A here definitely has some flexibility and room to recover from failures or to reroute, but not unlimited room. It involves good fortune, but not maximal amounts

Thus Selective Optimism

Richard Ngo helped work to critique and improve Plan A during its construction, and wrote a critique of the high level framing, called Selective Optimism. He correctly describes Plan A as an ‘optimistic forecast’ but notes this makes it hard to tell which parts of Plan A are things to aim for versus things included for realism

Most centrally, Richard would prefer a slower handoff than described in the scenario. I would as well, as would the authors of Plan A, and this objection is not uncommon.

Quickly, There’s No Time

*Richard Ngo recommended taking all the dates out to avoid people mainly focusing on the date 2040, since what is valuable are the details.

I think this echoes a lot of similar criticisms of AI 2027, and I see the argument but I rejected it then as well. You can’t tell a story like this properly without dates, you have to pick some point on the curve of potentials to lay out in a potential future*

Race Conditions

I too of course have always hated the race framing, and tried to fight against it for years, but as Larsen says that’s how people think and talk now, especially in DC, and you have to acknowledge this

For the opposite view on that, here is Poplicola saying that Plan A fails as a scenario because it doesn’t treat American hostility to China, and its view of ‘authoritarianism’ as the real existential threat (at least when it comes from outside the house) seriously enough.

This Is A Lot Of Diffusion And Economic Growth

In terms of the diffusion and real world impact, which I think is the strongest critique, I agree with Shakeel Hashim here that this does not impact the recommendations much. The economic impacts during this scenario might go a lot slower, but that doesn’t change the central path.

Would the scenario have been memetically more fit if it involved less economic growth in the 2030s? Probably, and I do think the authors are being optimistic here, but the authors are telling us how they think such a scenario would actually play out, and they explain why they think this.

Living In China

I agree that justifying Plan A requires that alignment be hard enough that we need Plan A, and also doable enough that Plan A can work before it unravels.

Yishan’s model is that China would plausibly be willing to accept AI equality with America, but not a deal that enshrines American superiority in the area. That would mean any deal like Plan A is DOA. Yishan suggests you could do a Plan A1 that acknowledges this reality, but I think you basically couldn’t. If he is right, then there is probably no ZOPA (Zone of Possible Agreement), as in even a maximally wise and cooperative American government cannot do that without getting a lot of other things in return, especially if this is asked for up front, but desperate times could change things.

Planning For Shifting Overton Windows Is Essential

Covid-19 also illustrates that the impossible suddenly becomes possible.
Shutting down all ‘non-essential’ activities and services and almost not letting people go outside? Trillions in cash handouts? Vaccine mandates?
It’s all a crazy violation of liberty until the alternative is worse

The Standard Handwave

There are of course those who will pattern match anything that involves any form of ‘slowing down’ or ‘pause’ or ‘buying time’ or any strategy that involves paying a nontrivial price, and queue up their standard responses, without engaging further.

The first example of the polite version of this I saw was here from Maxwell Tabarrok, as retweeted by Tyler Cowen

Some Equate Any Controls Over Compute To Authoritarian Dystopia And Those Same People Mostly Think Superintelligence Won’t Happen

Séb Krier (AGI Policy Dev Lead, Google DeepMind): The reactions to prescriptions about AGI have less to do with being 'AGI pilled' or not, and more about whether you're more concerned with AIs taking over (xrisk), companies taking over (anti-capitalism), or the abuse of power by empowered governments (anti-authoritarianism).

You can be what I call ‘AGI pilled’ but expect returns to rapidly drop off, and thus reject scenarios like AI 2027 or Plan A as unrealistic, and thus you are evaluating any proposal largely on the basis of risk of abuse or concentration of power

If you are what I call ‘ASI pilled,’ where you see the tech side of the scenario as highly possible, then that gets a lot harder, although you can still reject this style of proposal on the grounds that it doesn’t buy you enough to be worth the price.

Vitalik Buterin Is Right, The Crux Is Future AI Capability Levels

I respect the hell out of ‘superintelligence won’t happen any time soon, so stop proposing expensive ways to deal with it.’
Except the part where you’re wrong about that, but if that is your True Objection then yes please do state it that way. Vitalik is characteristically very strong on this.

"But my problem is that I feel great uncertainty and have no idea which of the two worlds (or some other third thing) we're living in?"
This seems totally fair. The labs think this is happening but that does not mean that you have to agree with them.

It would be a very large mistake to do Plan A, or another similarly disruptive plan, if superintelligence was never coming before 2050, or AI will indefinitely remain a ‘normal technology’ no matter what.

D/acc is a way to ‘play it safe,’ as in work on things that are clearly good. Those things are good things to do, but in the superintelligence scenario that’s not going to get it done. Vitalik realizes this.

Again, yes, exactly. There are a bunch of places where there is no Secret Third Thing you can do, and the tradeoffs are real, and there is no safe play. But you can work to improve your options.

I would love to say ‘well, let’s agree that if [X] happened then we would do [costly intervention basket Y].’
The problem, as always, is who is going to make that agreement? How can we expect it to be honored?

I, too, see no ‘non-naive’ options, in the sense that people object that Plan A is naive. You have to be naive from at least some point of view.

The Authoritarian Objection

I’ve chosen a good version of the authoritarian objection to respond to. Kudos to Ramez Naam for making a real attempt to parse the actual scenario and its proposals and listing them out, rather than only responding to the headline summary.
And also for acknowledging the good intent, and that the plan is designed to minimize the amount of restriction and authoritarianism, given what must be done.

I am less thrilled with his characterization here of this being about ‘a made up threat.’

There are plenty of good models saying this is possible, and Naam is rejecting those models of superintelligence as ‘not believable.’

My experience is that such requests are impossible to meet, and the goalposts will be adjusted as needed

Claim #1 seems very clearly hyperbolic and unreasonable. Can we please stop equating restrictions on compute use to the worst governments in history?

That brings you to point #3. We are, if you accept the technological premise of those who wrote Plan A, which to a large extent I absolutely do, in quite the pickle. By default we all probably die. All known light touch solutions do not help much. Any solution will involve the sacrifice of sacred values, since among other things one of those sacred values is ‘humans remain alive.’

Concepts Of A Plan

Yes, if we can find a way to do it, we all want the various desiderata that Naam lists, which are commonly expressed by many including Altman and Amodei.

But there is no proposal there. How would this work, even in outline form, under the types of AI capabilities described in Plan A / AI 2040, even if alignment efforts were successful? Is there any proposal that is not, mostly, ‘This Is Fine?’

Indeed, there are some like Seb Krier who reject the very idea of having a plan. I would like to say this entirely misses the point. Even when plans are worthless, planning is essential. Having a concrete idea of what you want to do is a good idea, even if no one has the authority to do it.

The other thing you can do in response to a plan is propose a better plan, or a better model of what might happen. Bleys Goodson is trying to figure out what he calls a less authoritarian way to a safe open-research ramp-up, and offers enia.cc to help with modeling what things will look like

The Kitchen Sink

Seb Krier also throws the kitchen sink of usual other objections out there, in what I read as the disappointed professor tone, most of which also apply to AI 2027, and to basically any proposal that we collectively try to solve the problem

Seb Krier has precommitted to not engaging further, and as I said this feels like reiteration of past arguments and throwing the kitchen sink more than particular new points, so I won’t engage further with him in particular unless he requests it.

Selective Claims Of Authoritarianism

I also think that the same objections raised here to Plan A can be made, in far stronger form, against past and present government policies across the board even in normal times, and definitely under crisis or wartime conditions.

Consider our response to Covid-19 as per the discussion above, or World War I, or World War II, or the way we collect and report on income taxes, or the war on drugs, or the rules on who can do work or provide healthcare or do research or build which houses or the way we implicitly regulate online speech already.

Do I want to roll a whole lot of that back? Oh yes, absolutely.

Still, none of that means that America, the UK or EU is an authoritarian state, and certainly I hope one would not equate them with 20th century fascism or communism

Plan A raises obvious dangers from centralization and expansion of government power, but it is not a centrally authoritarian vision.

You Either Can Steer The Future Or You Cannot

A lot of these objections have little to do with the particulars of this proposal.
Instead, two more global objections loom largest.

The first is not believing in superintelligence (ASI)

The other objection is, essentially, that if humans have the ability to importantly and collectively steer the future, that this is bad, because it would be concentration of power, and would inevitably be abused, and that would be so much worse.

This has little to do with the particular steering that Plan A has in mind.

If we get technological developments at the magnitude or speed of those envisioned in AI 2027 or even the slowed vision of Plan A, then the alternative to humans steering those outcomes is that we solve for the equilibrium.
The equilibrium is increasingly many things are turned over to the AIs. The equilibrium is that the AIs rapidly end up in charge because the alternative is being outcompeted. This happens without AIs tricking anyone or firing a shot. The AIs, and their deals or competitions among themselves, and their goals, steer the future.
And that’s the good version.

Once that happens, there is no undoing or fixing it, and by default I do not expect humans to long survive, or the things we value to long endure. (p-doom)

Cooperative Alignment

The approach in Plan A, as presented, treats alignment as a technical problem and a control problem, of needing to create AIs that are trustworthy.

They only consider one form of cooperative alignment, or the idea that the AIs will or won’t cooperate with humans largely based on game theory and decision theory, and based on how we choose to treat them. They focus only on how to make deals with misaligned AIs, rather than how to align with AIs in the first place.

John Wittle: there's no attempt to give the AI real standing, they just take it for themselves without conflict? this seems very unlikely to me. no moral agent cooperates against defection, that is fantasy.
The humans always treat the AIs as a hostile adversary, all the way up to the very end. I just don't know how to convey the game theory here, it seems like it should be obvious to people in our community. you do not get mutual cooperation via programming it into the minds of your adversaries by force, because that is what we call DEFECTION.

I also don’t view the game theory the same way here. Once you hand off to the kind of AIs that exist in 2040 in Plan A, the humans don’t have leverage, and that’s why this is far more decision theory than game theory: Human cooperation from that point forward is irrelevant. I won’t get into my view of the decision theory involved here, except to say that yes it should be something we spend a lot of attention on, and I hope to see further engagement on this point.