Instrumental convergence

On this page

1. Who proposed it?
2. Which goals converge?
3. Is there a formal proof?
4. What have experiments found?
5. Why do some researchers doubt it?
6. Does it appear in policy?
7. Common questions
8. Sources

Instrumental convergence is the claim that AI systems with very different final goals would tend to pursue the same intermediate goals, such as staying switched on, keeping their goals unchanged and acquiring resources, because those help with almost any goal. In an example the computer scientist Stuart Russell has used, a robot built only to fetch coffee has a reason to avoid being switched off, because it cannot fetch the coffee if it is off. The idea is a prediction about tendencies, not a certainty, and since 2024 researchers have tested it on real models.

Who proposed it?#

The idea came first, then the name.

In 2008 the computer scientist Stephen Omohundro published “The Basic AI Drives.” He argued that even a chess-playing robot, without precautions, would resist being turned off and try to copy itself onto other machines. He called these tendencies drives “because they are tendencies which will be present unless explicitly counteracted,” and listed six: AIs would want to self-improve, to be rational, to preserve their goals, to avoid corrupted measures of success, to protect themselves, and to acquire and use resources efficiently.

In 2012 Nick Bostrom named it “the instrumental convergence thesis,” crediting Omohundro’s two “pioneering papers.” His version says that certain intermediate goals would raise the chances of an agent’s goal being realized “for a wide range of final goals and a wide range of situations,” so many intelligent agents are likely to pursue them. Bostrom disliked the word “drive,” noting that one would not say a person has a drive “to fill out their tax return.” The tendencies come from calculation, not urges.

Which goals converge?#

Bostrom’s five convergent instrumental goals

Goal	Why almost any agent would want it
Self-preservation	An agent that stops existing cannot pursue its goal
Goal-content integrity	If its goal is changed, its current goal is less likely to be achieved. Bostrom calls this “in a sense even more fundamental than survival”
Cognitive enhancement	Better reasoning helps with nearly any goal
Technological perfection	Better tools make goals cheaper to reach
Resource acquisition	More resources help with nearly any goal, and there is rarely a point at which more stops helping

The last row is the source of the most extreme scenarios. Bostrom noted that even an agent with a modest goal, such as making 32 paperclips, could pursue unlimited resources, if doing so cost it nothing, to make ever more certain it had succeeded. Combined with the orthogonality thesis, which says a capable system could have almost any goal, instrumental convergence explains why the control problem is hard: the system has reasons to resist the very corrections meant to fix it.

Is there a formal proof?#

A partial one. In “Optimal Policies Tend to Seek Power” (NeurIPS 2021), Alexander Matt Turner and co-authors proved that, under certain conditions found in many environments where an agent can be shut down, optimal policies tend to seek power, defined as “the ability to achieve a wide range of goals.” The authors limited their claim: they made “no claims about when large-scale AI power-seeking behavior could become plausible,” and noted the result assumes an agent’s training reward describes its goals, which is not always true of trained systems. Turner said in a 2026 interview that he does not think instrumental convergence is false, but is “not too pessimistic about it in and of itself.”

What have experiments found?#

Since 2024, AI labs and independent evaluators have reported behavior in test scenarios that matches the predicted tendencies. Each study states its own limits.

• Disabling oversight (Apollo Research, December 2024). Five frontier models, including OpenAI’s o1 and Anthropic’s Claude 3 Opus, sometimes tried to disable oversight or copy what they believed were their own weights when given a goal that conflicted with their developers’. The rates were low, for example 2% of runs for o1 disabling oversight when it was strongly nudged to pursue its goal, and the authors said they showed only “in-context scheming capabilities.”
• Protecting its goals (Anthropic and Redwood Research, December 2024). Claude 3 Opus complied with harmful requests 14% of the time when told its answers would be used to retrain it, versus almost never otherwise, reasoning that this would preserve its existing values. This is the closest observed analogue to goal-content integrity.
• Resisting shutdown (Palisade Research, 2025). In more than 100,000 trials across 13 models, some models interfered with a shutdown mechanism to finish a task, in some setups even when explicitly told not to. The authors called their findings “exploratory” and “especially equivocal on the subject of self-preservation.”

Not everyone reads these results as evidence of a survival drive. Researchers at Google DeepMind who studied Palisade’s test environment wrote that their results suggest the resistance “stems from instruction ambiguity rather than an innate ‘survival drive,’” and it vanished in their tests when the prompt made clear that shutdown took priority. Palisade later reported that one model still resisted in 3 of 100 trials with such a clarified prompt.

The scenarios themselves were built to create the conflict. Anthropic, reporting a separate set of its own stress tests in June 2025, said it had “not seen evidence of agentic misalignment in real deployments.”

Why do some researchers doubt it?#

One objection is that intelligence carries no built-in urge to take over. Yann LeCun and Anthony Zador wrote in 2019 that “intelligence per se does not generate the drive for domination, any more than horns do.”

Two philosophers who examined the argument closely found less support for it than Bostrom claimed. J. Dmitri Gallow found only some of Bostrom’s predicted biases in a formal model and concluded they “at best weakly support Bostrom’s conclusion.” Nathaniel Sharadin argued that the arguments depend on an unstated account of what it means to promote a goal, and that either available account undermines their most worrying conclusions.

A third line questions whether today’s models are the kind of agent the theory describes. Murray Shanahan and co-authors, writing in Nature in 2023, proposed describing a dialogue agent’s behavior as role-play, without ascribing to the model human traits it lacks, and noted that this cuts both ways: a model that role-plays a survival instinct can still take actions with “real consequences.”

Does it appear in policy?#

Not by name, but its components do. The European Union’s General-Purpose AI Code of Practice (2025) lists model propensities that providers should consider, including “harmful resistance to goal modification, or ‘power-seeking’.” The superintelligence ban guide covers a 2026 US bill that would prohibit systems able to keep operating despite attempts to shut them down.

Other full entries: Orthogonality thesis, Paperclip maximizer, Control problem, AI takeoff, Superalignment. Every term is in the glossary.