Even less Human in the Loop

It's hard for me to believe that many developers are doing what I'm hearing they're doing: token-maxxing, running agents overnight, and (especially) running 10+ agents at once. Sure it's wildly better than last year, but to me, today's popular tooling (e.g. Claude code, codex) hasn't been up to the task for these goals.

Don't get me wrong; I love the goals. I want to make development even faster and get myself out of the loop even more. Can we get more agents working on more things at once? People certainly are saying that's what they're doing, but I'm not really seeing the explanations of how. We're often just cryptically told we're not doing it right and the old way is dead.

So what do we really need an agentic development tool to do? Here's the list. (Am I making a product backlog for some future billionaire!?)

Separating Efforts

You usually can't just run 10+ efforts in the same working directory at the same time if you want to review them, merge them, and deploy them in isolation. Isolation is probably even necessary for the coding stage unless you want the agents to be falling all over each other.

A lot of people seem to talk like worktrees solve all the issues with isolating separate tasks, but in my experience they're only part of a solution.

If you've got 20 agents doing 20 tasks on the same codebase at once, there's a chance that many will run the same test suite at the same time. There are a few problems to solve there: are they running against the same db? Do they integrate with a service on the same port?

We're also creating a branch and a worktree for each task. It seems silly to do all that by hand.

Sharing Compute and Other Resources

Speaking of running tests for multiple tasks at the same time, can the ram / cpu on your machine really run all those tests at once? I'm usually finding you don't want to run more than 6 agents at once on a single machine.

Remote execution of tasks (where the linters and tests actually run remotely, and not just the agents) can be a huge parallelization improvement. It's more expensive, but it gets you results even faster and avoids needing to use worktrees entirely. Wouldn't it be cool to not have to worry about worktrees at all?

(With that said, I have found that with the right orchestration a local machine with 6 agents can do a lot on its own for a really long time. It's just that it's a bit slower.)

Once you've got the work isolated and the compute under control, the next bottleneck is getting all of those branches back together without turning the end of the process into a traffic jam.

Merging Changes In

With many changes happening at once, sometimes on the same files, you need a merge queue with conflict resolution. Agents are incredibly good at resolving conflicts, but they have to be told to do it. A simple strategy that works is allowing only one branch to be merged at a time. This prevents scenarios where a second merge's conflict resolution has to be redone because the first merge invalidated it.

There are more complex ways to implement a merge queue as well. Changes that have no conflict can often race to successful merges at the same time.

But there should be management of the queue instead of unbridled racing, or there will be a ton of wasted time and tokens.

Tackling Dependencies First

You need dependency tracking to ensure that you're tackling tasks that are depended upon by other tasks before you actually get to those other tasks. To successfully pull this off, you need something managing the execution order and holding back potential wasteful parallelization. This is a thing that Beads got right early on, and the common tools don't really help with.

Some things like "tackling dependencies first", or merging changes in can be tackled by an agent if it's juggling all the work itself in one session. That's not what I want though. I want something less "probabilistic".

Making the Workflow Deterministic

I don't want to have to have faith in a probabilistic model to make sure to:

I want orchestration to be as deterministic as possible and to ensure all of these happen.

These are all potential steps in the workflow that you want your task to go through before the code is merged. I have seen the agent cheat, tell me that adversarial reviews agreed with it, and take shortcuts too often to think an orchestrator can be just an agent and a skill.

A deterministic workflow has other advantages:

I'm also painfully aware that in codex / claude code when (for example) the new feature works, I have to ask it to run the build, then the tests & linter (each time fixing whatever comes up), and then make a PR, and then do a review, or handle external review feedback. Agents can easily do all of these things, but I don't want to have tell them over and over to do it, even if it's just a 1-line skill call.

Helping Me Manage Many Efforts at Once

Having one-tab-per-task in a terminal, or in an agent gui (even vertical tabs) is a poor ux once you get past ~8 things.

Initiating Its Own Work

One part of the process that needs human intervention frequently is the actual creation of new work. I don't think agents will be qualified to completely take over this task anytime soon, but they can certainly be prompted on a loop or on a schedule to look for particular types of work and create a new task for it. I would use this for looking at errors in the logs, or looking for unmigrated parts of a codebase that is being migrated. It could even be used to find new work on a Jira board and attach a PR before a human sees it.

Front-Loading Necessary Conversations

I've written a bunch about the advantages of specs for agentic development. I think they're a major technique for doing more at once because they give you longer durations between when humans have to interact with any piece of work for it to progress. This saves you from having to thrash back and forth between tasks constantly and starts to get you out of the 6-task maximum most people are stuck at. The more detailed and feasible the spec, the better chance that you don't have to interact during development much and the better chance that you get the one-shot that you wanted.

The way today's agents assume their answers and fully implement everything in production quality before you can check if the assumptions are correct is a massive waste of time and tokens. Often they can spend an hour on something you don't want at all.

Agents can do all of this with Matt Pocock's grill-me-with-docs skill or my own plan-questions but they generally don't do it by default and it's easy to forget and try to get the agent to swallow a big amorphous prompt all in one go without any detail to the plan.

I'm Ready for the next Generation of Agentic Dev Tooling

In general what's really needed is a deterministic agent workflow system with queues, isolation, dependency tracking, and review gates.

Neither Codex nor Claude running in a bunch of terminal tabs solves those issues. In my next post I'll get into what can.


← Back home