AI Model Proves Existing Software Works Correctly

Anthropic has used its Claude model to mathematically prove that existing software code behaves as intended. The company announced the work on Wednesday, describing it as a method to increase trust in software without relying solely on traditional human review. This approach automates a process that has historically required extensive manual effort from specialists.

Formal verification, which proves software runs without error in all scenarios, has been expensive and rare. Engineers once required 20 person-years to verify seL4, a small operating system kernel. Because of this cost, the technique was largely reserved for critical systems like Airbus avionics. Anthropic's Frontier Red Team partnered with Theorem to apply this method to common open-source libraries.

The team tasked Claude with verifying code from 123 libraries within Google's OSS-Fuzz security project. Over several weeks, the model generated approximately 180 million lines of mathematical proof code. Claude did not write new software. Instead, it converted crash tests into mathematical problems and analyzed compiled machine code to construct the proofs.

A separate program called Rocq automatically checks the validity of these proofs. Anthropic states the process requires almost no human reviewers. The researchers describe the proof generation as fully autonomous, although they note that humans at Theorem built the underlying tools and formal foundations.

During the verification process, Claude found hundreds of bugs. The researchers permitted the model to fix these errors. Many of the bugs would likely have survived conventional testing because they only appear under specific conditions, such as unusually large input files or specific access sequences. When verifying a widely used PDF reader, the team found as many security bugs in one week as are typically reported for that software in an entire year. In another case, Claude identified a logical error that had remained unnoticed for more than decade.

Anthropic clarifies that formal verification does not guarantee the software is error-free. The proofs hold only under specific conditions: a particular processor architecture, single-threaded programs, and inputs smaller than 64 MB. Real-world deployments often use different hardware or configurations, which may fall outside these proof parameters.

Cost remains a barrier. Individual proofs consume hundreds of millions of tokens, and the company has not released total project expenses. However, a smaller test run showed that proofs can be reused. Efficiency is improving, with Opus 5.5 solving half of the proofs using about 40 percent fewer tokens than its predecessor, Opus 5.

Theorem's roadmap targets critical infrastructure libraries, including glibc, OpenSSL, and cURL. Anthropic expects AI-driven formal verification to become as practical as standard testing. The work shifts the focus from whether AI can code to whether it can prove that code is safe.