This post outlines how to use Node Version Manager (NVM) with yarn and Nix shells. I’m pretty new to Nix but have been using it tentatively for about 3 months since getting a new company laptop, I took the opportunity to try out something very different and hopefully learn something along the way.
I have used NVM for a long time but since starting my Nix journey hadn’t needed to run a specific node version beyond a major one. At this point I was using Nix for some things (like switching between major node versions) and falling back to brew and global installations of other tools when I couldn’t figure out the Nix pathway for the desired outcome. I still am, but slightly more heavily weighted towards Nix than I was.
First I’ll define some things so that you have some idea of what on earth I am talking about.
- What is Nix?
- What is a Nix shell?
- What is Node?
- What is Node Version Manager?
- What is Yarn?
- Why did I have to figure this out?
- Using NVM with yarn and Nix shells
- Summary
What is Nix?
Nix is a cross-platform package manager that utilises a purely functional deployment model. A Nix package describes how it should be built and the exact state of its required dependencies. Nix uniquely hashes each dependency and links the package to its declared dependency through some ninja symlinking. Nix provides the ability to describe and build reliably reproducible state for an environment.
What is a Nix shell?
Similar to a Nix package, a Nix shell is describes all of the packages that should be installed in a (mostly) isolated shell that Nix will build for you. Nix shells provide the ability to create development environments with just the dependencies a project requires, or to use packages that you don’t want to install globally. Think of Nix shells as a different approach to what a Docker container can do for isolated, reproducible development environments
What is Node?
Node is an asynchronous event-driven JavaScript runtime released in 2009 that is based on the V8 Javascript engine most famously used by the Chromium project and it’s offshoots (eg. Chrome). Node allows javascript projects to be built and run on any major platform by bundling a whole swathe of platform specific bindings for OS operations.
What is Node Version Manager?
As with any piece of software Node has had plenty of different versions built and released since its inception. Ordinarily running multiple versions of the same software in a single environment is always a little tricky. Node Version Manager (NVM) is a tool that makes that relatively simple to do with Node.
What is Yarn?
Yarn is a package manger for Node/javascript projects, the project does not have to be intended to run on node (eg. frontend projects) but does need to be built in a node
environment.
How did I get here?
I was asked to work on a yarn based project and created a shell.nix
file that looked like this to create a development environment (irrelevant parts removed):
{ pkgs ? import <nixpkgs> {} }:
with pkgs;
let
in mkShell {
buildInputs = [
nodejs-12_x
yarn
];
}
Nix was able to build the shell fine but when I tried to build the project with yarn
I got the following error:
error @cumulusds/aws-cloudformation-wait-ready@0.0.2: The engine "node" is incompatible with this module. Expected version "^12.13.0". Got "12.22.6"
error Found incompatible module.
It turns out the project requires a very specific version of node, 12.13.1
. I didn’t have nvm
enabled at this point and had wanted to use Nix shells to create the development environment. But time is precious when I’m working so after not finding too much straight forward advice on Google I checked out the source for the Nix node-12_x
package. Whilst nixpkgs
has derivatives for each major Node version, they install the latest minor/patch version, which makes sense. So, using the node-12_x
Nix derivation was not going to work.
I removed node
from my Nix shell and enabled nvm
on my machine. With no node
being built into the shell the node installation available in my global environment should be used by yarn
. I ran nvm use 12.13.1
and built my Nix shell. My shell.nix
file now looked like this:
{ pkgs ? import <nixpkgs> {} }:
with pkgs;
let
in mkShell {
buildInputs = [
yarn
];
}
I built the the Nix shell and tried to build the project with yarn
again but got the following error:
error @cumulusds/aws-cloudformation-wait-ready@0.0.2: The engine "node" is incompatible with this module. Expected version "^12.13.0". Got "14.17.1"
error Found incompatible module.
The version of node
I asked nvm
for is not being used. I ran nvm ls
and the 14.17.1
is installed on my machine so I deleted it and tried again. Same error! I looked for an underlying node
installation from brew
in case it was confusing matters? There was one from brew
, which I deleted. The only possible version of node now in $PATH
should be 12.13.1
managed by nvm
.
I built the the Nix shell and tried to build the project with yarn
again. You might be able to guess what happened next, same error:
error @cumulusds/aws-cloudformation-wait-ready@0.0.2: The engine "node" is incompatible with this module. Expected version "^12.13.0". Got "14.17.1"
error Found incompatible module.
I’m pretty sure at this point that node 14.17.1
is not on my machine. My next step was to remove node
, nvm
and directories where nvm
kept node installations, nuking my machine of node. I built the the Nix shell and tried to build the project with yarn
again. Same error:
error @cumulusds/aws-cloudformation-wait-ready@0.0.2: The engine "node" is incompatible with this module. Expected version "^12.13.0". Got "14.17.1"
error Found incompatible module.
I go and have a cup of tea to rethink the problem.
The problem is that in my Nix shell yarn
is finding a node installation and yet there is no node
installation on my machine. Additionally, when I do have node
on my machine the output of node --version
and yarn node --version
will differ within the Nix shell.
By the time I’ve finished my cup of tea I think I know what is going on. When Nix builds a package from a derivation it links the built package explicitly to its declared dependencies, this allows two packages in the same environment to depend on different versions of the same dependency. If one of the buildInputs
for my Nix shell declares a dependency on node
then when it executes a node
command it will call the node installation it declared as a dependency, not the one available at the top level of the shell.
Looking at the buildInputs
of my shells the first one worth checking for a node
dependency was yarn
. Sure enough, the source for the yarn
derivation declared nodejs
as a dependency (the latest node version). This meant that version of node
managed by nvm
in my global environment would not be found by yarn
in my Nix shell.
Using NVM with yarn and Nix shells
If you want to use NVM with Yarn and Nix shells then you must modify the yarn
derivation within your Nix shell definition. Some more information about overrides and how they are intended to used within Nix is available here
The desired outcome is to not have a node
installation built into the Nix shell so that NVM’s managed installations are picked up from outside of the shell. To achieve this I modified my shell.nix
file to look like this:
{ pkgs ? import <nixpkgs> {} }:
with pkgs;
let
in mkShell {
buildInputs = [
(yarn.override { nodejs = null; })
];
}
As the yarn
derivation was listing nodejs
as a dependency I needed to override that, effectively to nullify it and stop Nix building it into the shell. This would mean that when yarn
executes a node
command the external node
installation would be used, in this case the one managed by NVM.
Summary
I have found the learning curve with Nix pretty steep and I think I’ve only scratched the surface so far. As for using NVM with Nix shells, that was really helpful in beginning to transition to using Nix shells to orchestrate isolated development environments for my projects.
If you are using Nix and need very specific versions of Node for a project, know that using NVM with yarn and Nix shells is possible. With minimal change to your Nix shells to use a familiar workflow that incorporates NVM I found to be a good compromise whilst still getting the hang of Nix.
Get in contact
If you have comments, questions or better ways to do anything that I have discussed in this post then please get in contact.