The Agent Is Brilliant at the Task and Blind to the System
ProductBrain's tree view has to react fluidly to state, including state an agent is driving remotely. Building the effects to support that felt fast at first. It wasn't.
Twelve auto-expand effects. Added one at a time, over several weeks. Each correct. Each tested. Each solving a real problem in the tree view.
They also collectively broke the component in ways that were nearly impossible to trace.
The effects fired in unpredictable order and tripped on each other. A thirteenth to fix the cascade would have made it worse. The solution was an intent-based API that replaced all twelve with explicit state transitions. One day to build. The tree has been stable since.
The agent built every one of those twelve effects without once suggesting the approach was becoming dangerous.
The agent doesn't hold the system
This isn't a criticism. It's a description of how the agent actually works.
It optimises locally. Give it a task and it executes, cleanly, with gusto, every time. What it can't do is hold the system in memory across sessions. It doesn't know about the eleven effects before this one. It doesn't notice a pattern compounding. It never says this is getting complicated, maybe we should rethink the approach.
Builders who've shipped with agents know the harder question: what happens after fifty features, when the clean solutions have compounded into a mess the agent can't see because it never held the whole picture.
The human's job isn't to write the code. It's to notice when the code needs to stop being written and start being designed. This is a skill, not a setting. No tool installs it.
The pattern worth keeping
Let the agent build fast. Accumulate velocity debt intentionally. Intervene with architecture when the debt costs more than the speed.
This sounds reckless until you see the alternative: designing the perfect abstraction before you have any evidence of what the problem actually is. The agent gets you to the failure point in days instead of months. The rewrite takes a day instead of a sprint. You end up in the same place either way, but one path gives you earned understanding the other never does.
You can't know the right abstraction until you've seen the wrong one fail enough times. The agent makes that feedback loop fast enough to be useful.
The second layer: the agent defaults to the easy architecture
There's a subtler problem that shows up after you've identified the issue, and it's a trap for those who haven't thought about why the agent builds the way it does.
The agent has no problem with complicated. Feed it ten thousand lines of tightly coupled code and it navigates without losing the thread. It doesn't need the cognitive handholds that architecture principles were designed to provide.
There's an old principle that a good module hides its complexity behind a simple interface: the caller knows as little as possible, the implementation handles the rest. It exists because humans need protecting from complexity at the call site. The agent doesn't, so it doesn't bother. (Ousterhout calls these deep modules, and it's worth an afternoon if you haven't read it.)
What it produces instead: wide interfaces, shallow modules, everything coupled because coupling requires fewer decisions upfront. The caller knows everything. The implementation hides nothing. It's code that's legible to the agent and progressively less legible to the humans, and the future sessions, that follow.
When we decided to replace the twelve effects with an intent-based API (push complexity down into the tree, simple interface, tested in a harness), the agent's first proposal still had the orchestrator managing the tree's internals. But it looked cleaner than what we had. It was the same disease at a lower dose.
Architecture with an AI agent should be negotiation, not delegation. You set the principle. The agent proposes. You push back. It tries again. The right design emerges from the friction, not from the first suggestion.
You have to insist on the approach where the caller knows nothing. The agent will keep proposing the approach where the caller knows everything until you do.
What actually changes
There are two modes of work when building with an agent, and conflating them is expensive.
In the first, you're accumulating. The agent is building fast and you're watching what emerges. This is real velocity, but you're also collecting signal about where the design problems actually live. You're not being careless. You're getting data you can't manufacture any other way.
In the second, you step back. You look at what's there. You identify the principle the agent can't infer, the architectural intent that should govern the next phase, and you hold the line until the design reflects what you understand about the system.
The transition between them isn't always obvious. It feels like slowing down. It's the opposite.
Stepping back is the work.
No tool does the part this post is about. Seeing the system, knowing when to stop building and start designing: that's the work, and it stays yours.
But you can't do it from inside the weeds. When the agent loses the why and ships the wrong thing, your day goes to checking its work instead of doing yours, and the altitude this whole post is about is the first thing you lose.
That's the tax ProductBrain takes off your plate: the product's decisions and intent live as a model the agent reads, so it stops drifting and you stop re-steering. It won't see the system for you. It keeps you free enough to do that yourself.
Related: The Fifth Door Is Still Work: the full breadth of what shipping with agents actually requires. You Signed Up to Build Something: the skills that carry over when the tools change. A Philosophy of Software Design: John Ousterhout on deep modules, referenced above.