9 minutes
Nix and NixOS: a retrospective
Welcome dear reader to 2023!
If you’re reading this, like me you made it one more time around the great merry-go-round some people like to call orbiting our parent star. As methods go for demarcating the slow but relentless progression of time and the inevitable heat death of the universe, it’s not a bad one. 🤷
I hope this last year was good for all of you, or at the very least not a particularly shit one. As for the coming year, I can only hope we all spend more of it laughing than we do crying.
Unless you’re trying to learn Nix, in which case hang in there buddy, it does get better eventually… 😏 Which brings me to the topic of today’s post.
Looking back
As you may have noticed I talk a lot about Nix and NixOS. They say write what you know, and in that vein I’ve been writing posts about what I’ve been doing day-to-day over the last few months, which has inevitably involved a lot of Nix. That hasn’t always been the case.
In fact, I was first introduced to Nix around June of 2021, and would say that I’ve only been seriously using it for a little over a year.
So with last week’s blog post about NixOS services still needing some time in the oven, I thought I’d take this opportunity to look back at my experiences adopting Nix and NixOS and talk about my experiences so far.
How it all began
I was working on a client project in the summer of 2021, and we reached that stage where we needed to start bringing up staging and production environments etc. To help us do this we brought in Numtide, a DevOps consulting company that our client had heard about through the Nix community.
At this point I had never heard of Nix, but the client had and was interested in using it to build out our infrastructure.
So Jonas and Florian came onboard and over the next few weeks these files ending in .nix started appearing in our monorepo.
To begin with I didn’t pay much attention, I was too focused on delivering features and had filed Nix under “ops related” in my head. Over time I needed to interact with our new environments and update packages and so on, and that required learning more about Nix.
What I found intrigued me, and it quickly became apparent that Nix wasn’t just some “ops tool”, nor was it just a build system or a package manager. For me Nix and NixOS represented a paradigm shift, a fundamentally different approach to software and operating systems.
The ideas of reproducible builds and declarative system configurations wasn’t something I had encountered before. And they are ideas that I think have intrinsic value to anyone working in the software industry.
Over the course of the next few months I continued to dabble, making minor changes to our setup here and there. It didn’t take long though for me to come to the decision that the only way I was going to really get to grips with Nix and NixOS was by taking the plunge and making it my daily driver.
So I did.
Taking the plunge
It was easy enough to get a basic Gnome desktop environment up and running. And when it all just worked, it was great. I loved the idea that my system was declarative and reproducible.
But it didn’t take long for me to start having problems, and I would find myself in a situation where I couldn’t help myself. My desktop machine now ran on magic. I didn’t really understand how it all worked, and I was surprised by how difficult it was to learn.
In those early days I leaned heavily on the guys from Numtide, after all it was their fault for showing me Nix in the first place 😉 But not everyone has that luxury.
Besides, I prefer understanding what I’m doing. I’m weird like that…
So why is it so difficult to get to grips with Nix and NixOS?
Nix the language, NixOS the OS
I think before going any further it’s worth distinguishing between Nix the language and NixOS the operating system.
As a language, Nix is pretty straightforward.
It’s dynamically typed, lazily evaluated and all functions (or lambdas) take only one argument. Objects are called
attribute sets, basic types like arrays and strings etc. are all there, and you can bring in functions from other files
using the import keyword.
It was created with one purpose in mind:
It only exists for the Nix package manager: to describe packages and configurations as well as their variants and compositions. It is not intended for general purpose use.
Through Nix the language, NixPkgs came to be. It’s a package repository so big and well maintained that when comparing it next to the alternatives you have to adjust the scale.
NixOS then is the next logical step on from NixPkgs, a Linux distribution based on Nix and NixPkgs, that allows for a modular and declarative approach to defining your system configuration.
Instead of manually editing various configuration files, you describe what you want with Nix:
{
users.users.alice = {
isNormalUser = true;
home = "/home/alice";
description = "Alice Foobar";
extraGroups = [ "wheel" "networkmanager" ];
openssh.authorizedKeys.keys = [ "ssh-dss AAAAB3Nza... alice@foobar" ];
};
}
A nice side effect is that your system becomes fully reproducible (in theory), and if done right, portable to new machines or servers with little adjustment.
Why so difficult to learn?
Like with anything new, I am very conscious of the Dunning-Kruger effect. So what follows is a summary of what I found and continue to find difficult about getting to grips with Nix and NixOS.
Your mileage may vary.
Difficult to introspect
As everything in Nix land is lazily evaluated, I slowly found the only real way to know what was going on was to fire up the repl, load my config and start exploring.
In the early days I would just yolo and run nixos-rebuild switch. If it failed I’d have a look at the output, make an
adjustment, rinse and repeat. But even with the repl, it’s not a great feedback loop.
And whilst there are language servers for Nix that help with syntax and highlighting, I’ve yet to find a way of being able to “click-through” into the source code for library functions or to auto-complete what options may be available in a module.
Instead, I continue to spend a lot of time ripgrep-ing through my own config, NixPkgs and the source code for any extra modules I may have brought in. There are some helpers like NixOS options and Noogle, as well as some cli tools I have come across, but on the whole this is a disjoint and fractured experience that adds cognitive load.
In the end, if you want to understand what’s going on you just have to read the source code. And there’s a lot of it.
A lack of guided solutions
Ask a hundred Nix developers “what’s the best way to deploy a Nix system remotely?” and you’ll get a hundred different answers. Ask again about how to manage and maintain a multi system configuration, and you’ll get another hundred answers.
In my experience the same has been true for various issues I’ve encountered along the way (don’t ask about flakes… 🔥).
Now this isn’t necessarily a bad thing. You could argue it’s a reflection of just how expressive and flexible Nix and NixOS can be.
But for someone new trying to learn the first way of doing things, I feel what is missing is a more structured and guided introduction.
In the past, for example, I have helped people transition into software development, and the first thing I told all of them was this:
I’m going to show you how I do things. It’s not by any means the only way.
When you’ve learned enough doing it my way that you can question why, that’s when you should start looking at the alternatives.
But in the meantime, just do it the way I show you.
When starting out learning any new language or framework, or anything in general really, you don’t know what you don’t know, and you aren’t really equipped with the context and understanding required to evaluate your options.
So in this respect, I think the documentation for Nix and NixOS doesn’t go far enough, and needs to have a more thorough on-ramp for new users that encompasses more of the common use cases that they will encounter, providing them with an initial way of solving those problems until they have acquired enough knowledge to be able to evaluate the alternatives for themselves.
It’s just hard
When you consider the scope of what Nix and NixOS is addressing, is it any wonder that it’s just not easy to learn?
Sometimes a complex problem requires a complex solution, and in this respect I think Nix is hard to learn simply because it’s solving many difficult problems all at the same time.
Whilst I might think there are certain ergonomic improvements that can be made to improve things, I can’t see the intrinsic complexity of what Nix is doing ever being reduced to a point where it can be considered “easy to learn”.
It’s just not possible.
Will I keep using it?
Fuck yeah!
It’s been a longer road than I might have liked, but in my humble opinion Nix is a force-multiplier when it comes to system configuration and administration.
I continue to discover new and interesting ways that Nix can make my life easier and more productive.
Don’t get me wrong, I’m not naive to think that there aren’t more bumps in the road ahead. I fully expect to flip a table or two over the course of the next year.
But in the end, ultimately for me at least, it’s worth it.
Would I recommend Nix?
That’s a complicated question.
Adopting Nix is a long term investment. It’s not something you just pick up, and I would be hesitant about recommending it to a friend without first stressing the long road they have ahead of them.
In a professional setting, you have to consider how willing any potential client or organisation is to ensure that adequate training and support is provided, and to ensure that institutional knowledge is developed and retained.
In my case, the client was already aware of Nix and had made an informed decision about introducing it. Without this buy in, I would be hesitant to recommend Nix despite the benefits it could bring.
That being said, over the course of the last year there have been real efforts made to help improve the documentation, and more recently there is renewed interest in improving various aspects of the developer experience and helping bring Nix to the masses:
I hope to see such efforts continue and help bring down the barrier to entry.
Summary
It hasn’t been an easy road getting to grips with Nix and NixOS, but then again, things worth doing rarely are. Being introduced to Nix has fundamentally changed how I do my job, and in some ways how I view software in general.
For the time being, I’m happy to see where this road takes me, and I’ll continue to share what I learn along the way.