Parallelizability in Agentic Development

With agents, automated coding often seems like a solved problem. The natural next step seems to be that we should get more agents doing it at the same time.

I think we're certainly in a magical time for software development, and I do want to investigate this problem: how do we make more things happen at the same time? But the frame matters: the goal is finishing faster, not starting faster. Parallelization is only useful when it reduces the actual time to production. That's what this post will focus on, and in my next post, I'll go deeper on the "how".

Let's get clear on the importance of this problem

Before we get into parallelizability, I do want to say that I doubt that increasing parallelization of efforts will get us much productivity in most scenarios. The coding side of software development is 10x faster than ever before (at least!). The bottleneck is almost certainly elsewhere (e.g. are you sure you're even building the right thing? If you build it, will they come??).

So this is probably a local-maxima type optimization topic. I still want to think about it though. It's still going to be useful at times.

Should we even be trying this?

Because software development is often pretty complex, it's common for us to oversimplify and assume that parallelization is always an optimization. We don't see the cases where there's no advantage at all, or when it actually slows us down and causes unnecessary effort. If we're playing the game of "looking busy", or we're afraid to say no, we might try to take on many things at once and end up with far less actual productivity.

This is the first trap: measuring activity instead of throughput. The user (and an intelligent business) does not care about how many PRs you created; it's about what value you've actually delivered to real users in production.

Properties of Parallelizability

A set of changes is more parallelizable when there's very little interplay between the changes, and very little interplay between the changes and the humans involved. Specifically, the changes need to be free of the following aspects:

Inter-dependencies

First of all, in order for two agents to effectively work independently on two tasks at the same time, one of the tasks can't depend on the other one. The dependent task will ultimately be forced to land in production after the dependency if it's going to work at all.

This has always been a common blind spot in software development, and with coding almost instant, there's rarely a good reason to start anything before its dependencies are satisfied. Again: the goal is finishing faster, not starting faster. Starting the dependent task early only helps if it can still finish earlier after the dependency lands. Otherwise, you've just created more state to manage.

Code Conflict

One of the most common ways that a given piece of work can't be implemented at the same time as another given piece of work is when both pieces of work need changes to the same code. Branches and worktrees can give you a false sense of security that this is fine and parallelizable. They actually do very little to ensure parallelization though because ultimately you have to land your changes on trunk.

This is the second trap: treating isolated workspaces as isolated outcomes. They are not. The merge point is where the real dependency shows up. If two agents make incompatible changes, one of those changes may need to be significantly rewritten. As the cost (time and tokens) of resolving the conflict approaches the cost of writing that change to begin with, the case for parallelizing gets weaker and weaker.

SIDE-NOTE: It's worth noting now as well that token cost is a new factor to consider when deciding whether something is worth parallelizing. If the user has to wait another hour for the feature, but it costs nothing token-wise, what should we do? There's not a single answer for all scenarios.

The Need for Human Attention

Finally, we humans are still in the loop for many aspects of software development, usually at the very least for final review and approval. Additionally, many valuable kinds of work will require a human to review work early and often and will require adjustments to the plan along the way.

This is the third trap: assuming the agents are the only constrained resource. They usually are not. Exploratory or experimental work can't always be turned into a perfect spec and handed off to an army of agents. It requires iteration. I think this is usually the most valuable type of software development -- the kind where you're building something novel and won't imagine the perfect solution until you've iterated on it a few times.

That iteration puts humans back in the loop, which is fine, but a human has limits on how many things they can be available for at the same time effectively. It's a wild amount of context-switching. Most of us have tried and seen our own limits.

In the teams and users I’ve talked to, people often seem to hit a practical ceiling around six (some more, some less) concurrent agent tasks if they're using the common agents in typical ways. There's really only so much multi-tasking a developer can do at once effectively. Some days I personally find myself doing much less if I'm really iterating and exploring. It's a pretty stark contrast compared to what you see on Twitter, with people talking about using 20 agents at once.

These issues aren't new

Agentic development isn't bringing new problems for parallelizability. These have always been problems, and for decades people have tried to parallelize in the face of them anyway. What's changed is the economics.

When coding was slower, it was easier to justify starting more work early. Maybe some of it would be blocked, some of it would conflict, and some of it would come back for review at the wrong time, but the cost of keeping people busy was still often worth paying. With agents, "just working serially" is so blazingly fast that the coordination costs show up much sooner. A conflict that used to be a tolerable tax can now be larger than the work itself.

So the question isn't "Can agents work in parallel?", or even "Will it increase the number of PRs?". It's "Will parallel work reduce the actual total time to production?".

Now you may say "It's fine -- I let the agents deal with all of that," and there's a certain degree to which that is indeed possible, in my experience. But agents creating conflicts and then resolving them is much more expensive (in tokens at least, and sometimes in time / effort) than just sequencing the work correctly to begin with. And there are certainly times when I don't review an agent's code or test its changes, but those times are much rarer for shipped codebases with many active users. Letting many users be the first line of defense against defects is often just not a smart move.

So we need to parallelize selectively: when tasks are independently valuable, touch separate areas, require little shared context, and can be reviewed without overloading the human bottleneck.

Alright, so in my next post, I'll get into potential solutions, and how to actually break through the "sub-10 simultaneous agents" wall.