Effective Technical Improvement
One of my favourite ways to tackle tech debt is to fix it as I work through product requirements. There are some great advantages to it:
- It automatically prioritizes tech debt burndown in areas that you’re probably going to touch again soon, which is where it’s most valuable.
- It doesn’t require blocking any product development flow. It just slows it down (probably imperceptibly).
- It doesn’t even require conversations outside of engineering.
I'd hazard to say this is probably considered a best-practice, so people making technical improvements this way are in good company. I call this "the organic method" because improvements are happening naturally, or "organically" as other changes occur.
There are some downsides though (especially with large codebases with many affected developers):
- It hides the actual cost. Is it really better for product development to be imperceptibly slower? Wouldn’t it be nicer if costs were more explicit and obvious?
- It’s a lot easier to do tech improvement and product development separately. Doing two things at once is almost always more complicated.
- It’s easier to find patterns, better solutions and shortcuts for even fairly mechanical technical improvement work if you’re focusing only on that technical improvement work for some period of time.
Usually the biggest downside is that it’s slower.
In practice, I always find that it’s much much slower than you’d think. Here's a graph of a Javascript to Typescript conversion effort that I've been tracking for the past 11 months:
There are 2 small steep declines here that show the efforts of single individuals for short periods of time, but otherwise this graph (spanning almost a year) is a 64-file improvement out of 223 files in 11 months. At that rate, the effort will take 3.5 years.
I've tracked a number of similar efforts over the last year and the results are similar. My previous experience with organic improvement in large codebases feels pretty similar too: Without specific mechanisms to keep the conversion going, it naturally slows (or worse, stops).
Why does it matter if it’s slower?
Maintaining the incomplete state of the conversion is sneakily expensive:
- It's harder for newcomers to learn the right patterns when multiple exist
- Engineers need to remember all the ongoing efforts that are underway and always be vigilant in their work and in their code reviews of others work
- Diffs are easier to understand when they don’t try to do too many things at once
- Copy/pasta programming, forgetfulness, and uneducated newcomers lead to contagiousness; propagation of the wrong pattern instead of the right pattern
- When you’re really slow, you’re even more likely to have multiple of these efforts underway at once, compounding the complexity of fixing them
- If you’re slow enough and not actually doing cost-benefit analysis, patterns can be found that are “even better” than the ones underway. This is how you end up with even more ways to do the same thing and a smaller number of engineers that find joy in working in that codebase.
Most importantly though, if there’s really value in paying for that technical improvement, why not pay it sooner rather than later? Ironically, most of the least productive (and least fun) codebases I’ve seen are because of people making numerous actual improvements but then leaving them only partially applied. Good intentions without successful follow-through can easily make the code worse.
For larger technical improvements (ones that affect too many files to pull off in a week or less) you want to make sure that:
- You have a vague idea of the cost and you’re actually making an improvement that you think will be worth the cost.
- The timing for doing it now is right (and there isn’t something higher value you could do instead)
- You actually have a plan that converges on total conversion in a reasonable amount of time instead of something that just leaves the codebase in an inconsistent state for an extended period of time.
- The goal, the timing and the plan are generally approved by your teammates (even if unanimity is impossible)
Once you've got those 4 factors in place, you're probably better off in the long run if you capitalize on the improvement as quickly as possible. You probably don't want to cease all product development for a huge amount of time to do it, or send one developer hero off to fix it all, but you'll probably want to come up with something better than organic improvement too, if you really care about that improvement.
Tags: technical debt, refactoring
← Back home