Mutation Testing on Stuart Forrest

Using Stryker Runner VS Code Extension in monorepos

Fri, 08 Nov 2024 22:23:21 +0000

This post is about an update that I’ve made to the Stryker Runner Visual Studio Code extension I built enabling it to work much better with monorepos. For more information about the extension please see this post.

What is Stryker?
What is mutation testing?
What is VS Code?
Stryker Runner: what’s changed?

What is Stryker?

The Stryker project is a collection of libraries for languages including C# , Scala and Javascript that are used to run mutation tests against your codebase.

What is mutation testing?

Mutation testing is the programmatic changing or rewriting of parts of your code (a mutant) and validating whether the existing unit tests fail. The idea is that they do fail (killing the mutant), that is the theory anyway if you want to kill 100% of mutants.

Unit testing frameworks (like Jest for Javascript) can tell you how much of your code is covered by unit tests but this really just means which code gets executed under the tests. Unit testing frameworks cannot tell you how much of the desired behaviour is adequately tested and this information is a part of what mutation testing attempts to solve. The Stryker docs have a brief intro on this as well.

What is VS Code?

Visual Studio Code (VS Code) is free and widely used IDE developed by Microsoft. VS Code has access to an official marketplace of extensions that are used to add additional functionality, integrate libraries like Stryker and even add additional language support. There are also unofficial marketplaces that can be used instead of, or as well as, the official one, e.g. the open source marketplace Open VSX.

Stryker Runner: what’s changed?

The Stryker Runner extension has been useful for engineers that I have introduced it to and for those who have discovered it through other channels. Recently though we have started working with monorepos and have found that the experience using the extension was not quite as good as when working on single project repositories.

What was the problem?

When attempting to run Stryker against a file or code selection in a monorepo Stryker was more often than not unable to locate a relevant node_modules folder that it could symlink into the Stryker sandbox. This was not an issue if you choose to mutate in place.

When preparing a sandbox Stryker works its way up the file tree from the current working directory from where the stryker command was run looking for node_modules folder. Unfortunately, the extension had been built to execute commands from the root of VSCode Workspace that the target file belonged to. In a monorepo there may or may not be a relevant node_modules folder in the root of the Workspace depending on how the dependencies are being manged.

How did we solve it?

I’ve recently released an 0.0.47 release of Stryker Runner that changes its behaviour so that it no longer uses the root of the VSCode Workspace as the working directory from which to run the stryker command. Instead it locates the nearest ancestor package.json file to the target on which Stryker is going to be run and uses that as the current working directory.

This change is intended to ensure that when the Stryker Runner extension is used and Stryker works its way up the file tree from the current working directory looking for a node_modules folder it should always find the appropriate set of dependencies for the target.

The source for the Stryker Runner extension is available on Github.

The Stryker Runner extension can be downloaded from:

Get in contact

If you have comments, questions or better ways to do anything that I have discussed in this post then please get in contact via LinkedIn or email.

Stryker Runner VS Code Extension

Sat, 10 Sep 2022 18:12:21 +0000

This post is about the Stryker Runner Visual Studio Code extension that is intended to allow users to easily and selectively run the Stryker mutation testing library against Javascript and Typescript projects.

What is Stryker?
What is mutation testing?
What is VS Code?
Stryker Runner: the extension
Summary

What is Stryker?

The Stryker project is a collection of libraries for languages including C# , Scala and Javascript that are used to run mutation tests against your codebase. In the context of this post Stryker will always be referring to the Javascript/Typescript library, the extension does not currently support any other languages.

What is mutation testing?

What is VS Code?

Stryker Runner: the extension

Why bother writing an extension for VS Code now when the Stryker library for Javascript project has been around for 5 years at this point? Mostly because of the experience I have been having working with projects that heavily use Stryker mutation testing, I found that the developer experience was a little painful.

I think there is a lot of value in mutation testing your projects and I think Stryker is a great option. But because of the nature of what mutation testing is, what Stryker is trying to achieve and how it works Stryker can be a little painful to use sometimes. Depending on the size and complexity of the codebase, as well as the computing resources available, Stryker tests can be incredibly time consuming to run and lengthen the development feedback loop.

With Stryker you can run mutation testing against single files as well as against specific line ranges in files using the CLI. These approaches can really shorten that potential feedback loop during local development. The original idea of this extension was to surface this functionality into VS Code for developers.

How do I use it?

Against single files:
1. Right click on a file in the file tree in VS Code
2. This will present the context menu, the extension adds an item to the menu: Run Strkyer on file
3. Choosing this option will run Stryker against just that file in a new terminal
Against a line range in a file:
1. Select some code in a file in VS Code
2. Right click within the file
3. This will present the context menu, the extension adds an item to the menu: Run Strkyer
4. Choosing this option will run Stryker against the selected line range in just that file in a new terminal

Summary

A small group of developers have so far found the Stryker Runner extension to be useful, saving a lot of time in resolving the outputs from the mutation tests. If Stryker Runner sounds like something that might be useful in your workflow it would be great if you could give it a try. If there any issues, bugs, feature requests of any other kind of feeback please do get in contact or create an issue in the repo.

The source for the Stryker Runner extension is available on Github.

The Stryker Runner extension can be downloaded from:

Get in contact

If you have comments, questions or better ways to do anything that I have discussed in this post then please get in contact via LinkedIn or email.