What’s Different About Anthropic’s Use Of Fuzz Tests And Proofs?

🔍 Read the full analysis: What’s Different About Anthropic’s Use Of Fuzz Tests And Proofs? on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get the latest gadgets delivered free with Prime

• Fast, free delivery on millions of items
• Prime Video, Amazon Music and more included
• Member-only deals all year

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

Anthropic’s Frontier Red Team and collaborator Theorem reported on October 22 that Claude helped produce machine-checked security proofs for 239 fuzz-test entry points across 123 open-source libraries. The proofs establish that specified compiled programs do not crash on inputs under 64 MB, within the model and assumptions used; the team also says the effort found hundreds of bugs that existing fuzzing had missed.

The approach starts with an existing fuzz test, rather than asking researchers to write a complete description of what a library should do. The team treats the test as a specification: for every input below the stated size limit, the program must not crash. The libraries and tests are compiled into RISC-V executables and instrumented with AddressSanitizer, so certain memory errors trigger a crash.

Claude writes proofs in Rocq, a formal proof system, while Rocq’s checker verifies whether each proof follows from the specification. Theorem supplied formal models of RISC-V instructions and Linux processes. When a proof attempt fails, the failure can expose a potential bug; the team says Claude then patched some code, regenerated the specification and attempted a proof again.

Anthropic reports that the project produced about 180 million lines of Rocq proof over a few weeks. A typical proof was around 600,000 lines and covered a binary with a median length of 18,000 RISC-V instructions. Anthropic says it is publishing proofs that did not require code patches, previously unavailable trusted components, and specifications for tests it could not prove.

What Machine-Checked Proofs Add

Fuzzing checks many inputs, but it cannot establish that every possible input or execution path has been tested. A proof, when its specification and assumptions match the program being examined, can establish a property across all inputs in its defined range. That makes this work relevant to a familiar security problem: bugs can remain hidden even in heavily tested software, and some may be difficult to trigger through ordinary testing.

The reported findings illustrate the distinction. Anthropic says the proof effort found bugs requiring unusual input sizes or layouts, particular sequences of calls, or deeply nested data. It also reports a logic flaw involving a feature intended to restrict which files an application could load. The latter matters because not every security-relevant failure causes a crash, though the reported proof target was based on a fuzz test and its specified behavior.

The project also tests a possible way to reduce the labor involved in formal verification. Anthropic cites seL4 as an example of the high historical cost, reporting two person-years to build the microkernel and 20 person-years to verify it. The current results do not show that verification is now cheap in general: the source says individual proofs can consume hundreds of millions of tokens. They do show a research effort applying AI-generated proofs to many existing test targets, with a machine checking the resulting proof rather than relying on a human to inspect every line.

From Fuzz Tests to RISC-V Proofs

Fuzzing feeds programs large numbers of random or mutated inputs and looks for failures such as crashes. It is useful for finding bugs, but its results depend on which inputs and paths are reached. A test entry point gives the fuzzer a particular way into a library; it does not cover every way a library might be used.

Anthropic and Theorem’s reported method changes the question from whether a fuzzer found a crash to whether a formal model can prove that the specified entry point does not crash for any input within the stated size limit. This is a bounded, property-specific claim, not a general proof that a library is correct or safe. The formal models, theorem statements and Rocq checker are part of the basis on which the result depends.

Anthropic describes the effort as, to its knowledge, the first large-scale use of AI for formal verification of production software at the assembly-code level. That is the company’s characterization of the work. The binaries in the project were single-threaded, static RISC-V programs linked against musl. Most users encounter different builds, including x86 or ARM binaries linked against glibc, so the result should not be read as a direct proof about every deployed version of those libraries.

“The process “discovered hundreds of bugs.””

— Anthropic

Where the Proofs Stop

The reported results apply to the specific compiled binaries, test entry points and formal models used in the project. They do not by themselves establish that the libraries are free of all bugs, that every security property holds, or that alternate builds on common platforms have the same behavior. The no-crash claim is also limited to inputs under 64 MB and to the assumptions encoded in the specification and models.

Other details remain open in the source account. It does not provide a full breakdown of the hundreds of reported bugs, how many were confirmed or fixed by library maintainers, or the complete token and compute costs for the project. It says that proof generation can require hundreds of millions of tokens, while newer models and shared proofs may reduce that burden; the size and reliability of those savings are not specified.

Formal proof does not make the trusted foundation disappear. The result depends on the accuracy of the RISC-V and Linux process models, the exact theorem statements and the Rocq checker. Anthropic also identifies hardware, the operating system and the OCaml compiler used by Rocq as parts of the wider trusted setup. A proof can be mechanically checked and still establish only the property that was actually stated.

Publication and Verification Steps

Anthropic says it is releasing the unpatched proofs, trusted components that were not already public, and specifications for fuzz tests the system could not prove. Those materials will allow outside researchers to examine what was proved and review some of the assumptions and formal models behind the results.

Further evaluation will depend on whether researchers and library maintainers can reproduce the claims, validate the models and determine which reported bugs have been confirmed or addressed. The source material does not give a schedule for broader releases or say when the method will be applied to other architectures and production builds. For now, the report is a large-scale research result with a defined scope, not a blanket security certification for the libraries involved.

Key Questions

What did Claude prove?

Anthropic and Theorem say Claude produced Rocq proofs that 239 fuzz-test entry points in 123 open-source libraries do not crash for inputs under 64 MB, under the project’s specified assumptions and formal models.

Does this mean the libraries are bug-free?

No. The proofs cover a particular no-crash property for specific compiled programs and test entry points. They do not show that every feature works correctly or that every kind of security flaw is absent.

How is this different from fuzz testing?

Fuzzing tries many generated inputs and searches for failures. A formal proof aims to establish the stated property for all inputs in its defined range, rather than only the inputs reached during testing. That conclusion depends on the specification and model being accurate.

Do the results apply to software people use on ordinary computers?

Not automatically. The reported proofs cover single-threaded static RISC-V binaries linked against musl. The source says common x86 or ARM builds linked against glibc were not what these proofs directly covered.

What remains to be checked?

Researchers and maintainers can examine the published proofs and trusted components, check the formal models and specifications, and determine which reported bugs are confirmed and fixed. The full bug tally and project-wide compute costs remain unclear from the source account.

Source: ThorstenMeyerAI.com