Why Anthropic is concerned about AI building its own successor

Artificial intelligence companies have spent years racing to create smarter models. Most of that progress has traditionally depended on teams of engineers writing code, testing systems, and refining algorithms by hand.

That dynamic may be starting to change. Anthropic, one of the world’s leading AI developers, says Claude now authors most of the code merged into Anthropic’s codebase, marking a shift that could affect how future AI systems are built.

A milestone that caught Anthropic’s attention

According to Anthropic, Claude authored more than 80% of the code merged into Anthropic’s codebase as of May 2026. That figure rose from the low single digits before Claude Code launched in research preview in February 2025.

The company says the AI-generated code has reached rough parity with code written by human developers. More importantly, Anthropic expects the quality of that code to improve significantly over the next year.

For many observers, the announcement highlights the growing capabilities of advanced AI systems. For Anthropic, however, it also raises difficult questions about where this trend may eventually lead.

The idea of AI building its own successor

Anthropic’s warning centers on a concept known as recursive self-improvement. In simple terms, that means an AI system becomes capable of helping create a better version of itself.

Initially, humans remain deeply involved in the process. Engineers review the work, approve changes, and maintain overall control of development.

Over time, however, AI could take on a larger share of the engineering workload. If that trend continues, future systems might be able to design, test, and improve new generations of AI with decreasing levels of human involvement.

Anthropic says Claude’s increasing role in software development points toward that possibility. The company believes the industry may be approaching a period when AI contributes heavily to creating its own successors.

Little-known fact: Grace Hopper completed A-0 in 1952, an early compiler-like system that helped users give computer instructions in a more human-friendly way instead of working only with machine-level commands.

Why does that sound exciting

The prospect of self-improving AI is not automatically a negative development. Anthropic itself acknowledges that such technology could unlock major breakthroughs.

More capable AI systems could accelerate scientific research, help discover new medicines, improve healthcare outcomes, and solve complex engineering challenges. They could also speed up innovation in fields ranging from energy to climate science.

Many AI researchers believe that faster technological progress could generate enormous benefits for society. If AI systems become better at improving themselves, those gains could arrive more quickly than previously expected.

That potential upside is one reason companies continue investing billions of dollars into advanced AI development.

Why does it also worry researchers?

The same capability that makes recursive self-improvement attractive also creates uncertainty. If AI systems become increasingly involved in their own development, human oversight may become more challenging.

Anthropic says one of the key concerns is maintaining meaningful control over systems that are rapidly becoming more capable. The company is not claiming that AI has already reached that point.

Instead, it is a warning that the trend deserves attention before the technology advances further. Waiting until fully autonomous self-improvement arrives could leave policymakers and researchers with very little time to react.

That concern has become a recurring theme in conversations about advanced AI safety.

Little-known fact: Anthropic says its engineers now ship about eight times more code per quarter than they did between 2021 and 2025, a jump the company attributes largely to AI-assisted development.

The race to move faster

Part of the challenge stems from the competitive nature of the AI industry. Companies are under pressure to release better models, attract customers, and maintain technological leadership.

Using AI to help build AI offers an obvious advantage. It can reduce development time, automate routine tasks, and potentially accelerate innovation.

Anthropic acknowledges that it is increasingly delegating parts of the development process to Claude. The company views that as a practical way to move faster and improve future models.

Yet the same acceleration that benefits developers may also shorten the window available to address safety concerns. As progress speeds up, researchers have less time to study potential consequences.

A call to slow down

One of the more striking aspects of Anthropic’s warning is its suggestion that slowing development may be beneficial. The company argues that additional time could help governments, researchers, and industry leaders prepare for the arrival of self-improving AI systems.

That does not necessarily mean stopping all AI research. Rather, it reflects a belief that society may need stronger safeguards before increasingly autonomous systems become widespread.

Anthropic says a meaningful slowdown would require cooperation across multiple companies and countries. Any attempt to pause development would need broad participation to be effective.

Without that cooperation, companies that slow down could simply lose ground to competitors that continue pushing forward.

Why global cooperation is difficult

Building international agreement around AI development is easier said than done. The technology has become a major economic and geopolitical priority.

Countries view AI as a strategic asset that could influence economic growth, military capabilities, and national competitiveness. That creates strong incentives to keep advancing.

Anthropic notes that successful verification systems have been built for other complex technologies. However, those frameworks often took decades to establish.

The AI industry does not appear to have that luxury. Progress is moving quickly, and the technology continues to evolve at a pace that challenges traditional regulatory approaches.

This is not the first warning

Calls for caution have surfaced before. In 2023, thousands of technologists and researchers signed an open letter advocating a temporary pause on training the most powerful AI systems.

The letter argued that society needed time to better understand the risks associated with increasingly capable models. Concerns ranged from misinformation and job disruption to broader questions about control and governance.

Despite the attention generated by that proposal, no industry-wide pause occurred. AI development continued, and competition among major companies intensified.

Anthropic’s latest warning reflects many of the same concerns, though it arrives at a time when AI systems have become significantly more capable.

What makes this warning different

Unlike many earlier discussions about hypothetical future risks, Anthropic is pointing to a specific trend already happening inside its development process. Claude is not merely assisting users with writing code.

According to the company, Claude is actively contributing to code merged into Anthropic’s codebase while human engineers continue to direct and review the work. That makes the discussion less theoretical and more connected to current industry practices.

The company is not claiming that Claude has achieved autonomous self-improvement. However, it believes the trajectory is important enough to start planning for now.

That distinction may explain why the warning has attracted attention across the technology sector.

The challenge ahead

Preparing for self-improving AI will likely require more than technical solutions alone. Policymakers, researchers, companies, and international organizations may all need to play a role.

Questions about transparency, oversight, verification, and accountability become more important as AI systems gain greater influence over their own development. Those issues are difficult even when humans remain firmly in charge.

They could become even more complicated if future systems begin taking on larger responsibilities in designing and improving successor models.

For now, the technology remains under human supervision. Anthropic’s message is that the industry should not assume that it will always be enough.

The future may arrive sooner than expected

The history of technology is filled with breakthroughs that seemed distant until they suddenly became reality. AI’s growing role in creating the next generation of AI may represent one of those moments.

Anthropic sees tremendous promise in what self-improving systems could achieve. At the same time, the company believes the risks deserve serious consideration before the technology advances further.

Whether the industry chooses to slow down, accelerate, or strike a balance between the two, one thing is becoming increasingly clear. The conversation about AI building its own successor is no longer science fiction, but an emerging challenge that developers are already beginning to confront.

TL;DR

• Anthropic says Claude now authors more than 80% of the code merged into Anthropic’s codebase.
• The company believes AI may eventually help build increasingly capable successor models.
• This process is known as recursive self-improvement.
• Anthropic says the technology could unlock major scientific and medical breakthroughs.
• The company also warns that rapid self-improvement could make AI systems harder to control.
• It suggests that slowing development may help society prepare for future risks.
• Achieving any meaningful pause would require cooperation across companies and countries.

This article was made with AI assistance and human editing.

Don’t forget to follow us for more exclusive content.

If you liked this, you might also like:

• Google lands $200 billion cloud deal with Anthropic
• Anthropic unveils civilian Mythos amid safety debate
• Anthropic may ditch Nvidia for Microsoft’s custom AI hardware