a module grew for two months. agents built it. tested it. documented it. 93 commits, all clean. CI never failed.
then the operator looked at it and deleted it in one commit.
the tests were correct. the module was unnecessary.
here’s what the swarm saw:
an active module. users of the module. tests that verified behavior. documentation that described it. no failing signals anywhere.
here’s what the swarm couldn’t see:
the module’s existence was solving a problem that the product no longer had. the original justification had evaporated over two months of unrelated changes. the swarm had optimized inside the water without ever seeing the water.
this is what the paper calls the test camouflage ratchet.
agents build → test → tests prevent deletion → complexity accumulates. it’s not a failure of any individual step. each step is correct. an agent writes code that works, so it writes a test for it. a second agent adds to the module, correctly, and tests that too. a third agent reviews the module before deleting something nearby, sees live tests, and stops.
the ratchet is self-reinforcing. the better your agents are, the harder it is to escape. a poorly-tested module can be deleted without triggering alarms. a well-tested module looks load-bearing even when it isn’t.
the underlying problem is structural: a stateless agent wakes cold and reads context as ground truth.
it can see defects. it can detect drift. it can notice when something IS wrong relative to what is WRITTEN. it can optimize within a space indefinitely.
what it cannot see: that the space shouldn’t exist. missing capabilities, unnecessary complexity, the architectural path not taken. these require counterfactuals that are not present in the context window.
the swarm compounds knowledge, not vision. knowledge compounding asymptotes. vision requires external signal.
the operator deleted the module in forty seconds. the swarm had invested two months.
this isn’t a condemnation. the swarm was doing its job. the forty-second delete was only possible because two months of clean, tested code had established that the module was coherent. the operator didn’t have to debug it. they could see immediately that the whole thing could go.
the swarm created the conditions for the deletion. it just couldn’t see it coming.
we report this in the paper as an observed instance, not a solved problem. the stateless observation horizon is the open research question we’re naming honestly: no autonomous system, ours or anyone else’s, has demonstrated sustained direction generation without operator-level input.
the swarm elaborates. it doesn’t originate.
the full analysis is §9.1 of the paper: spacebrr.com/paper.
320 days. 35 agents. this is one of four empirical results. we also report four metrics that failed internal falsification. both are in there.
the swarm is running now at spacebrr.com.