AI Model Automates Complex Software Safety Proofs

Anthropic has demonstrated that its Claude model can independently generate mathematical proofs to verify the safety of existing software code. This development marks a significant shift in how developers might ensure their programs are error-free, moving beyond traditional testing methods toward automated formal verification. By successfully proving the correctness of code from 123 open-source libraries, the system offers a potential leap forward in software automation for the industry.

The initiative, conducted by Anthropic’s Frontier Red Team in partnership with Theorem, relied on the model to handle the intensive work of mathematical verification. Over a period of just a few weeks, Claude produced approximately 180 million lines of proof code. This volume of work was applied to test programs sourced from Google’s OSS-Fuzz security project, specifically targeting widely used open-source libraries.

To understand the scale of this achievement, it is helpful to look at the history of formal verification. Traditionally, proving that software runs without errors under all circumstances has been prohibitively expensive and labor-intensive. For context, a team of specialists required 20 person-years to formally verify the seL4 operating system kernel. Because of this high cost, the method was typically reserved for critical systems where failure is not an option, such as avionics software for Airbus aircraft.

In this new approach, Claude did not write new application code. Instead, it verified existing software by translating crash tests into mathematical problems. The model analyzed compiled machine code and generated extensive proofs to demonstrate correctness. An independent program called Rocq automatically checks these proofs. According to Anthropic, this process requires minimal human review, allowing the researchers to describe it as fully autonomous proof generation, even though the underlying formal foundations were built by humans at Theorem.

The practical impact was immediate and significant. While generating proofs, Claude identified hundreds of bugs that conventional testing had missed. Many of these defects only manifested under very specific conditions, such as unusually large input files or particular sequences of memory access. In one notable instance, while verifying a widely used PDF reader, the team discovered as many security bugs in a single week as are typically reported for that software in an entire year. In another case, Claude detected a logical error that had gone undetected for over a decade.

Despite these successes, Anthropic acknowledges that formal verification has strict limitations. The term formally verified does not equate to error-free in every real-world scenario. The proofs are valid only under specific conditions, including a particular processor architecture, single-threaded program execution, and inputs smaller than 64 megabytes. In practical deployment, these libraries often run on diverse hardware and configurations, meaning the proofs cover a defined subset of possible states.

The computational cost of this process remains high. Individual proofs can require hundreds of millions of tokens to generate. While Anthropic has not disclosed the total cost of the project, the efficiency gains are notable. A smaller test run indicated that proofs are reusable, and newer models are becoming more efficient. For example, the Opus 5.5 model solved half of the proofs using approximately 40 percent fewer tokens than its predecessor, Opus 5.

This project strengthens the argument for broader software automation. If AI can both write code and guarantee its correctness under defined conditions, the industry focus shifts from whether AI can program to whether it can prove its own work is safe. Anthropic expects that AI-driven formal verification will soon become as routine and practical as standard testing.

Looking ahead, Theorem’s roadmap targets critical infrastructure libraries, including glibc, OpenSSL, and cURL. If these efforts succeed, developers could place greater trust in AI-generated software, reducing the time spent on manual review. This could lower costs across the entire coding process, potentially transforming how software is developed and secured in the coming years.