Node on Stuart Forrest

Hugo pretty URLs - migrating from Lambda@Edge to Cloudfront Functions

Tue, 07 Sep 2021 05:33:23 +0000

This post is intended to share the process of migrating URL prettifying functionality for a static site hosted in AWS to Cloudfront Functions from Lambda@Edge and deploying them with Serverless.

Some background on my site
What are Cloudfront Functions?
What is the Serverless Framework?
Migrating from Lambda@Edge to Cloudfront Functions
Summary

A bit of background

This website is built using a static site generator called Hugo, deployed using the Serverless Framework and hosted in AWS using S3 for storage, Cloudfront as a CDN and up until recently Lambda@Edge. At build time all content is rendered into static HTML and CSS files, there is no client side magic that something like React provides. Hugo produces an index.html for every post in it’s own folder, I was using Lambda@Edge to transform more friendly paths like /some-fancy-post-name into /some-fancy-post-name/index.html that S3 understands so that the content can actually be found. It is nothing fancy, just appending index.html to any path that doesn’t have it.

What are Cloudfront Functions?

Cloudfront Functions, whilst not a successor to Lambda@Edge, are an evolution of them. Whereas Lambda@Edge allows fully fledged lambda functions to run within a regional edge cache (AWS region) closest to the request origin, Cloudfront Functions run right at the edge where the content is ideally going to be served from. In terms of Cloudfront architecture you cannot get closer to the user. There are some restrictions to Cloudfront Functions you have to bear in mind like having a limited runtime, execution time, memory allowance, package size and that they are javascript only with no network access.

What is the Serverless Framework?

The Serverless Framework is an infrastructure-as-code tool that simplifies the defining and deployment of mostly event driven serverless architectures. When used with AWS it is an abstraction over Cloudformation and does a lot of the heavy lifting to create and wire up resources like lambdas, queues, apis and events. As it compiles down to Cloudformation it can support other resources too but these are defined in regular Cloudformation syntax.

Migrating pretty URLs from Lambda@Edge to Cloudfront Functions

Migrating the Lambda@Edge that handled pretty URLs to a Cloudfront Function was pretty straight forward. The code itself went from this:

exports.handler = (event, \_, callback) => {
 const request = event.Records[0].cf.request;
 const indexPath = new RegExp("/$");
 const match = indexPath.exec(request.uri);
 const newURL = request.uri;

 if (match) {
 newURL = `${request.uri}index.html`;
 }

 request.uri = newURL;
 callback(null, request);
};

to this:

function handler(event) {
 var request = event.request;
 var indexPath = new RegExp("/$");
 var match = indexPath.exec(request.uri);
 var newURL = request.uri;

 if (match) {
 newURL = `${request.uri}index.html`;
 }

 request.uri = newURL;
 return request;
}

As you can see very little changed, sourcing the value for request was the only functional change to the code required. Some other changes were required because of the restricted runtime for Cloudfront Functions:

No exports allowed!
No let or const, not used good old fashioned hoistable var in a while

The feature availability in the Cloudfront Function runtime is a real jumble of pre-ES5 plus a bit of ES6, ES7 and beyond. And no exports! I was keen to unit test these functions and the lack of exports presented a challenge, I will write up how I went about that in a future post.

Cloudfront Functions code is also required to be inline in the CloudFormation template. This isn’t great from a testability and IDE syntax linting/hinting/etc points of view. Also, only one function per trigger is allowed which makes reusability of the code a challenge. For example with my development version of the site I expected to be able to use two Functions in a pipeline on the viewer-request trigger, one for HTTP Basic Auth and one for pretty URLs, then only pretty URLs in production. Alas, the code had to be duplicated, and this limitation already exists with Lambda@Edge.

For the inline code problem, I was fortunate enough to be using a Typescript file to define my Serverless config instead of yml so I just read an external file containing the full code for my Cloudfront Function:

fs.readFileSync("./lambdas/cfFunctions/prettyUrls.js", "utf8")

This worked like a charm but is there an alternative? Yes, use CDK, which handles reading the code in from a file for you (docs).

Summary

Migrating from Lambda@Edge to Cloudfront Functions was an opportunity to try something new from AWS and push some compute as close to the end user as possible. It went really well and the bulk of the time I spent doing it was spent figuring out unit testing (I wrote about that here) and a clean deployment strategy as opposed to writing the code and just getting it deployed.

For simple uses that manipulate the request/response objects without the need for external input then Cloudfront Functions are a better alternative to Lambda@Edge, otherwise Lambda@Edge isn’t going anywhere. For those simple use cases migrating between the two is pretty straight forward though deployment and a robust development lifecycle for Cloudfront Functions can have a little bit more complexity.

Get in contact

If you have comments, questions or better ways to do anything that I have discussed in this post then please get in contact via LinkedIn or email.

Using path aliases in VSCode in Javascript or Typescript projects with Jest and Webpack

Fri, 28 May 2021 11:33:23 +0000

This post might be helpful if you are looking to use path aliases within a javascript or typescript project.

What are path aliases?
Why use path aliases?
Configuring Typescript and your IDE
Configuring Jest
Configuring Webpack
Any other options?
Demo repository

What are path aliases?

Path aliases are the mapping of some kind of shorthand identifier to an import path (usually) within your project.

A relative import path like ../../data/types could use an alias like @shared/data/types.

In effect, whenever your tooling encounters the shorthand reference it will know where to look to actually import the desired module.

Why might you want to use them?

Improved readability

There is a cognitive load when importing relative modules in your code using explicitly relative paths. It will usually look something like this:

import { maxCacheAgeInMS } from "../../../config";
import { CompanySummary, DynamoDBQueryRecord, Product } from "../../data/types";
import chunkArray from "../../utils/chunk";

Whilst there is nothing wrong with this why not look at the alternative. What if, instead of a relative path to our shared folder being used every time we want to import an module from there we could use a fixed shorthand? Then it could look something like this:

import { maxCacheAgeInMS } from "@config";
import { CompanySummary, DynamoDBQueryRecord, Product } from "@shared/data/types";
import chunkArray from "@shared/utils/chunk";

In this example @shared is a path alias that has been setup to map to [projectRoot]/src/shared meaning that wherever we use it within our project our toolchain of choice will be able to find and import the same module.

I prefer the readability of this setup and find the cognitive load of reading it much smaller.

Encapsulation

When we use path aliases it can make it easier for us to refactor or change our project structure, if we wanted to move the location of our shared modules we can update the alias mapping in one place.

Whilst most modern text editors-cum-IDEs (VSCode included) can handle this for us by monitoring our project structures and updating imports for us when it changes it doesn’t always work as expected (especially in large projects/codebases) and often has some developer overhead of checking and confirming each change is valid.

Using path aliases within a project can help encapsulate this location logic within the codebase itself. On it’s own this is not the strongest argument for path aliases but builds on the benefits of increased readability.

What is the catch?

There is no such thing as a free lunch and whilst using path aliases comes with some benefits there is some setup and configuration required. There are some potential additional headaches in store for the tooling that is used around your project like:

Building/bundling, eg. babel, esbuild, tsc, webpack etc.
Testing, eg. jest etc.
IDE support, eg. VSCode, Webstorm etc.

This post focuses on the use of path aliases with:

VSCode
Jest
Webpack
Typescript

For the most part the configurations can be defined once and shared between these tools, this is great and reduces the potential headaches (and complexity) or setting up and maintaining path aliases in multiple places.

Configuring path aliases

Remember that path aliases are essentially a mapping between a shorthand reference and a qualified path. When configuring path aliases this is usually represented as a key -> value mapping that is observed by your tooling. The remaining questions are where does it go and what does it look like? We will walk through the answers to those now.

Typescript and IDE (our source of truth)

Typescript has the concept of projects and all your code must belong to a project, ordinarily this root of your repository represents the root of your project but not always. The Typescript compiler, tsc, looks for tsconfig.json files below the directory from which it has been run in or pointed at. Every directory in which it finds a tsconfig.json file is treated as the root of a separate Typescript project. For the purposes of this post we will assume you are working with a single Typescript project. Note that if you have multiple projects your aliases defined in a tsconfig.json are only valid within the scope of that project.

Whereas Typescript expects a tsconfig.json file to be present, pure Javascript projects don’t require an analogous jsconfig.json file. The concept of a jsconfig.json file was introduced by VSCode and modelled on the tsconfig.json spec used by Typescript. VSCode uses this file to, amongst other things, help Intellisense make sense of path aliases. Whilst it was introduced by VSCode it has been adopted or supported by other IDEs like Webstorm (since 2019.2).

The config

There is a paths property under compilerOptions in the tsconfig.json spec as you can see below. This is where our alias mappings should go:

{
 "compilerOptions": {
 "baseUrl": "./",
 "paths": {
 "@shared/*": ["src/shared/*"],
 "@config": ["config.ts"]
 }
 }
}

In this example you can see that you can point an alias to a specific file if you want to though usually a wildcard (*) is included that will be expanded to the remainder of the import path after the alias value. For example @shared/data/types would be expanded to src/shared/data/types at module resolution time.

There is also a baseUrl property under compilerOptions, if set the path aliases values are relative to this.

At this point the Typescript compiler and VSCode should quite happily be able to consume our path aliases within our project. VSCode may need a few minutes in a particularly big project, alternatively you can reload the project from the Command Palette.

Single source of truth

Our tsconfig.json or jsconfig.json will be used as the single source of truth for our path aliases within our project. How this is achieved for the rest of out toolchain varies from tool to tool but there are some handy helper modules we can use to make it relatively straight forward.

Jest

Unless otherwise specified Jest will assume your configuration is within your package.json file or a standalone jest.config.js file.

The Jest configuration property, moduleNameMapper, can be set and will inform Jest how to manipulate imports as required. This includes handling path aliases like those set up in the previous step. This property can look something like this:

{
 moduleNameMapper: {
 "^@shared/(.*)": "<rootDir>/src/shared/$1",
 "^@config$": "<rootDir>/config.ts",
 },
};

A glob/regex is used to match against import paths and mapped to one or more possible locations where these modules may be imported from.

I should note here that the moduleNameMapper configuration is not specifically for handling path aliases, it is intended for any use case where modifying imports in the test environment is desirable. For more information see the documentation.

Using our single source of truth

We are using our tsconfig.json or jsconfig.json files as a source of truth for our path aliases. With jest that are a couple of ways to do this.

If you are using ts-jest it comes bundled with a handy utility function that can be used to transform the alias config from your tsconfig.json into what Jest requires. Usage is as follows:

const { pathsToModuleNameMapper } = require("ts-jest/utils");
const { compilerOptions } = require("./tsconfig");

{
 moduleNameMapper: pathsToModuleNameMapper(compilerOptions.paths, {
 prefix: "<rootDir>",
 })
}

As you can see passing a prefix of “<rootDir>” is often required as that is Jest’s representation of the root of your project and the mapper utility makes no assumptions about prefixes for your paths.

If you are not using ts-jest or do not want to couple yourself to ts-jest in this manner there is a standalone npm module jest-module-name-mapper that does the job as well. Usage is as follows:

const { default: tsconfigTransformer } = require("jest-module-name-mapper");

{
 moduleNameMapper: tsconfigTransformer("./tsconfig.json")
}

// OR (as per the readme)

{
 moduleNameMapper: require("jest-module-name-mapper")("./tsconfig.json")
}

As you can see the usage is pretty much similar. The main difference is that jest-module-name-mapper does make some sensible assumptions about the prefix jest requires for your paths and also about the location of your tsconfig.json file. I’ve passed in the location explicitly for clarity but this would have been the assumed location anyway.

Webpack

If you are using Webpack then it too will need to know how to handle your path aliases. There is a webpack configuration property resolve which has a child property alias (as seen below, docs), this is where we can let Webpack know how to handle our aliases. As with jest there is a handy npm module (or two) that can use your tsconfig.json as the source of truth for your aliases. These are:

I prefer the latter as the former replaces the context of resolve.aliases making it difficult to add aliases in addition to those within your tsconfig.json. Ultimately choosing which to use should be a decision based on the needs and context of your use case. Below is the usage of resolve-ts-aliases:

import { resolveTsAliases } from "resolve-ts-aliases";

const config: webpack.Configuration = {
 resolve: {
 alias: resolveTsAliases(path.resolve("tsconfig.json")),
 },
};

How you are using Webpack in your project will determine whether this alias mapping will be required, but if you find that Webpack cannot resolve your aliases the configuration above should hopefully do the trick.

Other options?

Most Javascript projects of a certain size, especially anything being shipped out to the world, will have some build tooling. The common build tools have been covered here but if you have a project with no tooling, for example you intend to run your script with node index.js then the npm module module-alias can be used. It intercepts node’s module resolution process to resolve any known aliases. Unfortunately, whilst it can play nice with webpack it struggles to do so with Jest (because Jest bypasses node’s module resolution) and may present challenges with other toolchains. How to use path aliases in this way is out of scope of this post but check it out if this sounds like it might be useful.

Demo repository

There is a repository with a tiny demo project with path aliases configured and working with Typescript, Jest and Webpack available here.

Get in contact

If you have comments, questions or better ways to do anything that I have discussed in this post then please get in contact via LinkedIn or email.