I feel more and more that I am loosing control of what's happening under the hood. Agents move fast and chaotically. Are there frameworks, tools or governance solutions to give us back more control?
Agents are fast and powerful, and that is a double-edged sword. The volume you can produce is breathtaking, but they make far too much for you to review every line of output. Personally, I focus my limited attention on the places that give the biggest levers and put deterministic guards throughout.
First, recognize that most of what you want done is actually a sequence of steps, and those steps produce intermediate artifacts. And you can either review or gate artifacts as they flow through the process. The earlier you look, the more leverage you have.
I make software, which follows these steps: make a plan with tasks, make a design for each task, code each task, run the tests. I spend a lot of time on the plan, because mistakes here expand drastically once they hit code.
But it's not just all on me: I write scripts and use tools to make guarantees about the artifacts. For a plan, for example, I have a script that makes sure it has the correct structure. I also use another agent to review it according to my personal specification.
Every artifact has this, because the reliability of the final product of the pipeline is actually composed from all those checks.
Force the (planning) agent to plan a specification for anything you're doing. Ask the agent to break it down into small steps that can be isolated commits. Review these steps as you feed them to the (coding) agent. Work through them one at a time.
Actually look at the diff that's produced.
---
Basically, that's a clone of how I would work if I was the coder. I don't plan as well as agents, but I know what needs to happen better. But I work in small commits on isolated areas. The coding agent will screw things up if you have him winging it and making sweeping changes across a repo. So make it smaller, be specific, and maintain a traditional "small, non-breaking commit" workflow.
First, recognize that most of what you want done is actually a sequence of steps, and those steps produce intermediate artifacts. And you can either review or gate artifacts as they flow through the process. The earlier you look, the more leverage you have.
I make software, which follows these steps: make a plan with tasks, make a design for each task, code each task, run the tests. I spend a lot of time on the plan, because mistakes here expand drastically once they hit code.
But it's not just all on me: I write scripts and use tools to make guarantees about the artifacts. For a plan, for example, I have a script that makes sure it has the correct structure. I also use another agent to review it according to my personal specification.
Every artifact has this, because the reliability of the final product of the pipeline is actually composed from all those checks.
Actually look at the diff that's produced.
---
Basically, that's a clone of how I would work if I was the coder. I don't plan as well as agents, but I know what needs to happen better. But I work in small commits on isolated areas. The coding agent will screw things up if you have him winging it and making sweeping changes across a repo. So make it smaller, be specific, and maintain a traditional "small, non-breaking commit" workflow.