Using NVM with yarn and Nix shells

Wed, 29 Sep 2021 20:02:49 +0000

This post outlines how to use Node Version Manager (NVM) with yarn and Nix shells. I’m pretty new to Nix but have been using it tentatively for about 3 months since getting a new company laptop, I took the opportunity to try out something very different and hopefully learn something along the way.

I have used NVM for a long time but since starting my Nix journey hadn’t needed to run a specific node version beyond a major one. At this point I was using Nix for some things (like switching between major node versions) and falling back to brew and global installations of other tools when I couldn’t figure out the Nix pathway for the desired outcome. I still am, but slightly more heavily weighted towards Nix than I was.

First I’ll define some things so that you have some idea of what on earth I am talking about.

What is Nix?
What is a Nix shell?
What is Node?
What is Node Version Manager?
What is Yarn?
Why did I have to figure this out?
Using NVM with yarn and Nix shells
Summary

What is Nix?

Nix is a cross-platform package manager that utilises a purely functional deployment model. A Nix package describes how it should be built and the exact state of its required dependencies. Nix uniquely hashes each dependency and links the package to its declared dependency through some ninja symlinking. Nix provides the ability to describe and build reliably reproducible state for an environment.

What is a Nix shell?

Similar to a Nix package, a Nix shell is describes all of the packages that should be installed in a (mostly) isolated shell that Nix will build for you. Nix shells provide the ability to create development environments with just the dependencies a project requires, or to use packages that you don’t want to install globally. Think of Nix shells as a different approach to what a Docker container can do for isolated, reproducible development environments

What is Node?

Node is an asynchronous event-driven JavaScript runtime released in 2009 that is based on the V8 Javascript engine most famously used by the Chromium project and it’s offshoots (eg. Chrome). Node allows javascript projects to be built and run on any major platform by bundling a whole swathe of platform specific bindings for OS operations.

What is Node Version Manager?

As with any piece of software Node has had plenty of different versions built and released since its inception. Ordinarily running multiple versions of the same software in a single environment is always a little tricky. Node Version Manager (NVM) is a tool that makes that relatively simple to do with Node.

What is Yarn?

Yarn is a package manger for Node/javascript projects, the project does not have to be intended to run on node (eg. frontend projects) but does need to be built in a node environment.

How did I get here?

I was asked to work on a yarn based project and created a shell.nix file that looked like this to create a development environment (irrelevant parts removed):

{ pkgs ? import <nixpkgs> {} }:
with pkgs;

let

in mkShell {
 buildInputs = [
 nodejs-12_x
 yarn
 ];
}

Nix was able to build the shell fine but when I tried to build the project with yarn I got the following error:

error @cumulusds/aws-cloudformation-wait-ready@0.0.2: The engine "node" is incompatible with this module. Expected version "^12.13.0". Got "12.22.6"
error Found incompatible module.

It turns out the project requires a very specific version of node, 12.13.1. I didn’t have nvm enabled at this point and had wanted to use Nix shells to create the development environment. But time is precious when I’m working so after not finding too much straight forward advice on Google I checked out the source for the Nix node-12_x package. Whilst nixpkgs has derivatives for each major Node version, they install the latest minor/patch version, which makes sense. So, using the node-12_x Nix derivation was not going to work.

I removed node from my Nix shell and enabled nvm on my machine. With no node being built into the shell the node installation available in my global environment should be used by yarn. I ran nvm use 12.13.1 and built my Nix shell. My shell.nix file now looked like this:

{ pkgs ? import <nixpkgs> {} }:
with pkgs;

let

in mkShell {
 buildInputs = [
 yarn
 ];
}

I built the the Nix shell and tried to build the project with yarn again but got the following error:

error @cumulusds/aws-cloudformation-wait-ready@0.0.2: The engine "node" is incompatible with this module. Expected version "^12.13.0". Got "14.17.1"
error Found incompatible module.

The version of node I asked nvm for is not being used. I ran nvm ls and the 14.17.1 is installed on my machine so I deleted it and tried again. Same error! I looked for an underlying node installation from brew in case it was confusing matters? There was one from brew, which I deleted. The only possible version of node now in $PATH should be 12.13.1 managed by nvm.

I built the the Nix shell and tried to build the project with yarn again. You might be able to guess what happened next, same error:

error @cumulusds/aws-cloudformation-wait-ready@0.0.2: The engine "node" is incompatible with this module. Expected version "^12.13.0". Got "14.17.1"
error Found incompatible module.

I’m pretty sure at this point that node 14.17.1 is not on my machine. My next step was to remove node, nvm and directories where nvm kept node installations, nuking my machine of node. I built the the Nix shell and tried to build the project with yarn again. Same error:

error @cumulusds/aws-cloudformation-wait-ready@0.0.2: The engine "node" is incompatible with this module. Expected version "^12.13.0". Got "14.17.1"
error Found incompatible module.

I go and have a cup of tea to rethink the problem.

The problem is that in my Nix shell yarn is finding a node installation and yet there is no node installation on my machine. Additionally, when I do have node on my machine the output of node --version and yarn node --version will differ within the Nix shell.

By the time I’ve finished my cup of tea I think I know what is going on. When Nix builds a package from a derivation it links the built package explicitly to its declared dependencies, this allows two packages in the same environment to depend on different versions of the same dependency. If one of the buildInputs for my Nix shell declares a dependency on node then when it executes a node command it will call the node installation it declared as a dependency, not the one available at the top level of the shell.

Looking at the buildInputs of my shells the first one worth checking for a node dependency was yarn. Sure enough, the source for the yarn derivation declared nodejs as a dependency (the latest node version). This meant that version of node managed by nvm in my global environment would not be found by yarn in my Nix shell.

Using NVM with yarn and Nix shells

If you want to use NVM with Yarn and Nix shells then you must modify the yarn derivation within your Nix shell definition. Some more information about overrides and how they are intended to used within Nix is available here

The desired outcome is to not have a node installation built into the Nix shell so that NVM’s managed installations are picked up from outside of the shell. To achieve this I modified my shell.nix file to look like this:

{ pkgs ? import <nixpkgs> {} }:
with pkgs;

let

in mkShell {
 buildInputs = [
 (yarn.override { nodejs = null; })
 ];
}

As the yarn derivation was listing nodejs as a dependency I needed to override that, effectively to nullify it and stop Nix building it into the shell. This would mean that when yarn executes a node command the external node installation would be used, in this case the one managed by NVM.

Summary

I have found the learning curve with Nix pretty steep and I think I’ve only scratched the surface so far. As for using NVM with Nix shells, that was really helpful in beginning to transition to using Nix shells to orchestrate isolated development environments for my projects.

If you are using Nix and need very specific versions of Node for a project, know that using NVM with yarn and Nix shells is possible. With minimal change to your Nix shells to use a familiar workflow that incorporates NVM I found to be a good compromise whilst still getting the hang of Nix.

Get in contact

If you have comments, questions or better ways to do anything that I have discussed in this post then please get in contact via LinkedIn or email.

NVM on Stuart Forrest

Using NVM with yarn and Nix shells

What is Nix?

What is a Nix shell?

What is Node?

What is Node Version Manager?

What is Yarn?

How did I get here?

Using NVM with yarn and Nix shells

Summary

Get in contact