A few caveats worth mentioning right away:
- We continued adding features while converting. +
- We spent a bunch of time additionally working on making the service’s input types definable in Typescript with runtime validation (we convert typescript to json-schema during the compile step). This expanded the effort significantly, but it also allowed us to trust that type safety would hold at run-time.
- We additionally converted from Promises to async/await as we went.
Alright, Let’s do some math
In calendar time the conversion took about a year, but I estimate the actual developer effort was about 6 engineer-months. ++
The Importance of our Test Suite
During the conversion, we found surprisingly few defects in the existing code. It was common for a single developer to be working on conversion full time and not find any defects at all in a week.
I probably converted over 10% of the code myself and never personally ran up against any defect that was user-facing. We ultimately did find a handful of user-facing defects in the pre-conversion code though (maybe 5?) and most had to do with the input validation not being strict enough. Typescript was useful for helping us find those cases in a way that tests probably wouldn’t be (It’s rare for our automated tests to be testing bad input extensively).
The conversion process did in fact introduce test breakages quite regularly though (that would have been bugs). We used the “noImplicitAny” compiler flag to help us enforce using types everywhere. At the file-level, this is an all-or-nothing conversion strategy, so on a large or complicated file, conversion could take hours before you could run the tests again, and continually I was finding that even with a successful build, the tests would fail in dozens of different unexpected ways, all indicating new defects.
Let me repeat that for emphasis: the act of changing existing working code to add types introduced defects that only the tests caught. This probably happened in more than half the files that I converted.
Conversely, what we’ve accomplished with tests (and no type-checking) on this codebase over the last 5 years is pretty mind-boggling to me. We converted:
- From callbacks to promises
- From express 2.0 to express 3.0 to koa.
- From mongodb to Amazon Aurora for core data.
And throughout each of these conversions, we deployed multiple times per day. None of the conversions were “big bang”.
What I like (because I like Typescript):
Well I’m using VS Code now, instead of just vim, as one does when switching to Typescript. It’s more sluggish of course, but it comes with better in-editor support for Typescript and that means that:
- I can see a bunch of potential defects in-editor without running the tests at all. That’s fast feedback.
- Autocomplete is a lot smarter than what I’ve seen in any vim plugins.
- Documentation about different function interfaces or object shapes is available right in the editor.
That last one is really key for me and ties into a larger improvement than just those in the IDE. Typescript encourages us toward standardizing object “shapes”. With Typescript interfaces, we can say things like “Alright a ‘user’ will always look like this. It will always have these fields.” If some other representation has less fields, or more fields, and we really need that, we’ll have to face the pressure of adding an additional type. That’s pretty powerful and I really appreciate it. Before, just reading application code, I’d have to consult tests to see exactly what shapes and interfaces were supported or returned and there was no pressure to simplify and consolidate on just one shape. That hurts learnability and maintainability.
There are of course a bunch of places where defining types is a productivity drag. It’s certainly more code to write and more code to read, and for certain types of work, it can amount to just noise. When writing new code though, I do really appreciate how easily I can call on file-external classes/methods/functions/properties without having to jump around from file-to-file and figure out the proper way to do that.
Static Types and Quality
It’s pretty clear to me that we’re not really catching many more defects with Typescript compared to the effort that we’ve put into it.
We’re catching defects faster though with faster feedback, right? Well I don’t know about that either. We definitely are for certain classes of bugs. If a function that adds 1 and 1 together returns the string “2”, that will get caught much faster. There are definitely lots of typos and misspellings that are caught more quickly too. I’m enjoying that immensely and it’s a real productivity boost.
It’s not free though. Compile time is around 40 seconds for me (and worse for others). So if I want to actually test runtime behaviour I’d say I’m now at a disadvantage. Firing the server up and manually testing locally now takes almost a minute when it used to take less than 10 seconds. Even with incremental compilation and a warmed cache, there’s a ~10 second compile time, which makes doing TDD (my usual way of writing code) a lot slower because of the slower feedback cycle. So, many types of defects are faster to detect, but many types are also slower.++++
But I don’t have to write as many tests, right? I have no idea why anyone would think static-typing replaces certain kinds of tests. I was never testing types. Does anyone ever do that? I still write exactly the same number and types of tests. Testing hasn’t changed a bit.
If it sounds like you’re working in a scenario that is close to ours (a multi-year SaaS product), my recommendation is that if you have to choose between static-typing and tests, choose tests. To forego tests in this type of codebase is not only lazy but lazy in a way that actually leads to more work. It’s hard to watch developers avoid the tiny upfront investment of tests when I can see what it’s given us over the lifetime of this codebase.
You don’t have to choose either, so if you’re sure you want a statically-typed codebase, feel free to do that too. Typescript is not going to be a silver bullet though. The if-it-compiles-it-works approach is a recipe for long-term pain.
++ I’m pretty confident in that estimate because around 80% of the conversion was done by one engineer fully tasked with the effort.
++++ I can’t really totally blame Typescript for this. Our Typescript-to-JsonSchema converter is included in this compile stage, and it takes a significant chunk of the time. We really want to ensure that type-safety holds at run-time as well though. Additionally, I do think we should be relying on incremental compilation (with a watcher) more. It’s possible with more effort we can fix most of the delay.