Announcing Treefmt 2.0

I’ve been an avid user of treefmt, and it’s nix companion treefmt-nix, for a couple of years now. Whenever I start a new project, one of the first things I do is configuring treefmt with all the linters and formatters for the stack I’ll be using.

I just love being able to run treefmt and have a vast array of linters and formatters take care of cleaning up my project in one go.

But as much as I enjoy using treefmt, it has not been without its issues. Issues that I wanted to fix as a good citizen of the Open Source community, but which I didn’t feel capable of addressing given that the project was written in Rust.

That’s not to say I didn’t try (a little).

If you look through the V1 branch you’ll see a commit or two from me tackling a minor issue here or there. But if I’m being brutally honest, I wasn’t particularly motivated to improve my understanding of Rust and to try and make changes in place.

Goodbye Rust, hello Go?

It started as an experiment at the end of last year, just to see how far I could get re-implementing treefmt in Go with a few days of effort.

As it turns out, I was able to get pretty far.

And that only motivated me to keep going, playing around with new models and ideas. It was a joy working on such a tightly defined problem in which success is easily measured, unlike a lot of my client engagements which tend to be much more open-ended and fluid.

I don’t remember exactly when it was that I showed it to zimbatm.

I do remember being a bit sheepish, after all, I’d just gone and started re-writing his project for reasons that start with “I know Go better than Rust” and end with “Go just feels more appropriate, we don’t need a sledgehammer (Rust) for this” 🤷.

What surprised me was how open he was to moving to Go:

If it’s fun for you, go for it!

Fast-forward a few months, and we find ourselves here, with the release of Treefmt 2.0.

What’s new?

First of all, we have a shiny new website treefmt.com. 🙂

But more importantly, we have improved performance a lot.

By replacing the toml based cache file with boltdb and msgpack, and re-imagining how we apply formatters concurrently, execution times are up to 8 times faster, with sub-millisecond times for smaller repositories when the cache is hot and no files have changed.

❯ treefmt -c
traversed 100 files
emitted 100 files for processing
matched 51 files to formatters
formatted 0 files in 493.970818ms

❯ treefmt    
traversed 100 files
emitted 0 files for processing
matched 0 files to formatters
formatted 0 files in 972.811µs

For large repositories, such as nixpkgs, when running a sample config that first looks for dead nix code, and second formats it, we’ve seen non-cached execution times reduce from ~70 seconds to ~24 seconds, and cached execution times reduce from ~1.2 seconds to ~250ms.

❯ treefmt -c -u debug      
traversed 41687 files
emitted 41687 files for processing
matched 34568 files to formatters
formatted 19991 files in 23.693930325s

❯ treefmt -u debug   
traversed 41687 files
emitted 0 files for processing
matched 0 files to formatters
formatted 0 files in 258.40926ms

This means you can now safely configure format-on-save using treefmt on large repositories without significant UI delays.

In addition to improvements in performance, we’ve made it easier to ensure formatters are executed in a particular order for a given path, whilst retaining a high level of concurrency. In fact, we now guarantee that only one formatter will operate on a given file at one time.

# One CLI to format the code tree - https://git.numtide.com/numtide/treefmt

[formatter.deadnix]
command = "deadnix"
options = ["--edit"]
includes = ["*.nix"]
# lower the priority, the higher the precedence
priority = 1

[formatter.nixpkgs-fmt]
command = "nixpkgs-fmt"
includes = ["*.nix"]
priority = 2

The remainder of the differences between version 1 and version 2 are mostly minor, as we wanted to ensure that version 2 is a drop-in replacement for version 1.

In the future, we may decide to diverge in terms of configuration or cli args. If that happens, we’ll move to version 3.

Future Work

Now that we’ve gone to the trouble of re-writing everything in Go, and in the process knocking out some high-profile issues, we want to let things settle.

It’s important to us that we haven’t broken existing workflows, and if we have, we will be quick to get them resolved. Please drop an issue if you encounter a problem 🙏.

After things have settled, we will have a look through the backlog and see what new and interesting features we can start adding!