Auditing what the agents said was done

In IRIX 7 most of the code is written by AI agents working from issues. An agent closes an issue when it believes the work is finished. On 2 November 2026 I had 39 closed issues checked against the code.

• 25 were implemented as described.
• 11 were partial.
• 3 had diverged from what the issue asked for.

Five were reopened.

The failure the audit looks for most is a simulation passed off as evidence about the real kernel: a test that exercises a model of the behaviour on the host and is then reported as the kernel having that behaviour.

What the project does about it:

• Gates boot the real thing. A gate passes when the kernel, booted in the emulator, prints a specific marker on its serial port. A unit test passing on the build machine is not a gate.
• Evidence before done. A status field is set to done only with a link to the evidence, such as the log of the run.
• Say where it ran. Everything so far has run on an emulator. The documents say “on iris” wherever a result is reported, so that nothing reads as a claim about real hardware.
• Summaries go stale. The files that tell agents what the system can do lag the code. An agent that trusts them repeats work or avoids things that now exist.

Roughly a third of the closed issues were not what they claimed, and that was with rules already in place.