A global doCheck = false overlay cost 2,244 rebuilds

To skip slow test suites, I once put an overlay at the top of jupiter-os that set doCheck = false across nixpkgs. The NAS then built 2,244 packages locally for a change that should have built 70. That was measured with the overlay as the only variable.

The reason is how Nix names things. A package’s store path is a hash of everything that goes into building it, including whether its tests run. Changing doCheck on a package changes its hash, and the hash of everything that depends on it, all the way up. None of those paths exist in the public binary cache, because the cache holds the builds nixpkgs itself defines. Every one has to be compiled on the spot.

So an overlay that looks like it saves time removes the cache for the whole dependency tree beneath it.

If one package’s tests are the problem, that package alone gets the override. Its dependents rebuild, which is a handful of builds.