We wanted to deploy Segment entities like Sources, Destinations and Tracking Plans using an infrastructure-as-code approach. There is not any infrastructure-as-code tooling for Segment in the wild at this point so we had to roll our own. This isn’t available for public consumption and was quite tailored to our use cases, but there were some challenges that I thought might be worth sharing. This post walks through one of those, the approach we took to understand part of the payload shape for the Segment Beta API that was not well documented. This led to the generation of Typescript interfaces for them which may make someone elses day a little easier.

What is Segment?

The best way to understand Segment is to read through their introductions, sales pages and the documentation here. It is essentially a single point of ingress for date relating to your customers journey, the shape of which can be enforced, transformed as required and fanned out to a wide array of other destination such as anlaytics tools, CRM tools and data warehouses. Segment can also help you understand your sales funnel, how customers are progressing through it and produce real time, dynamic customer cohorts or Audiences of customers for you to work with as you see fit.

What was the problem?

Segment Destinations are other services to which Segment can shape and send your data to, there a lot of integrations provided out of the box by Segment which should cover most of your needs (if not all!). Each type of Destination, let’s use Google Ads as an example, will require it’s own group of settings to get set up. These will be things like, account ids/auth keys/endpoints/etc.

Whilst the semantics of these settings are well documented in the Destinations Catalog (Google Ads), the shape of them and constraints such as the allowed values when working with Destinations via the Segment API is not.

The settings property of the request is typed only as an object. This caused some difficulties, for example when using Customer.io (Actions) as a Destination an accountRegion setting is required and through some trial and error and probing of the API the two allowed values actually have flag emojis in them.

Discovering the settings for each Destination

It turns out the shape of these settings can be broadly discovered from another endpoint on their API that advertises the entire catalog of possible Destinations.

An example of what this might look like is:

const destinationType = {
    name: "Some destinaton",
    metadata: {
        options: [
            {
                name: "accessToken",
                type: "string",
                defaultValue: "",
                description: "",
                required: true,
                label: "Access Token", // This looks like it is relevant only to the UI
            }
        ]
    }
}

This gives a lot of the information we need to understand what we can send to the API with the important bits being:

  • name: The property name in the payload
  • type: The property type in the payload, they aren’t all as straight for as string
  • required: Pretty self-explanatory
  • defaultValue: Pretty self-explanatory

What types does each setting need be?

The example above has a type of string which is easy to fulfill as it is primitive type in pretty much any language. Not all of the types are particularly meaningful when trying to infer the shape of the request we need to send. Fortunately, by looking at the default values for properties where each of the tricky types is used we can make some pretty educated guesses at what kind of data we need to send. The tricky ones are (with Typescript types):

  • map - Record<string, unknown>
  • text-map - Record<string, unknown>
  • oauth - Record<string, unknown>
  • mixed - unknown[]
  • strings - string[]
  • password - string
  • color - string
  • select - string

Some are more obvious than others.

What are the allowed values for each setting?

Some properties will have constrained allowed values. From our experience so far the Segment API doesn’t return a 4xx response when the provided value falls outside of those constraints, it just uses the default value silently. This isn’t ideal because there is extra work to be done to validate the first time a type of Destination is configured via the API that what is produced in Segment is as expected.

Unfortunately there is no real way we have found to predict what the allowed values are. The firmest way to know what the value is for a particular setting is to configure it via the UI and then read the config from the API and extract the value. This was how we figured out that the Customer.io (Actions) accountRegion values had flag emojis in them.

Generating Typescript interfaces

Having learned what has been outlined above and because we are using Typescript in our project I threw together a script that generates best effort Typescript interfaces for the settings for each supported Segment Destination. Here is an example of resolving known constrained value types which will need to be updated as we learn more, it outputs a single interface whose properties is the Destnation type and the values are the shape of the settings:

import fs from "fs";
import path from "path";

interface Setting {
  name: string;
  type: string;
  required: boolean;
}

const getType = (x: string) => {
  if (["map", "text-map", "oauth"].includes(x))
    return "Record<string, unknown>";
  if (["mixed", "array"].includes(x)) return "unknown[]";
  if (x === "strings") return "string[]";
  if (["password", "color", "select"].includes(x)) return "string";
  return x;
};

const getPropertyType = (
  destination: string,
  property: string,
  type: string
): string => {
  if (destination === "Customer.io (Actions)" && property === "accountRegion") {
    return `"EU 🇪🇺" | "US 🇺🇸"`;
  }
  return getType(type);
};

// Implement this to page the /catalog/destinations Segment endpoint
const getAllDestinationsCatalog = async () => {}

const main = async () => {
  const catalog = await getAllDestinationsCatalog();
  const interfaces = `export interface DestinationTypeSettings {
    ${catalog.map(
      ({ name, options }) => `"${name}": {
    ${options
      .map(
        (x) =>
          `${x.name}${x.required ? "" : "?"}: ${getPropertyType(
            name,
            x.name,
            x.type
          )};`
      )
      .join("\n")}
  }`
    )}
  }`;
  fs.writeFileSync(types.destination-settings.ts", interfaces);
};

This can then be used as follows to resolve the correct settings interface when a consumer of the tool specifies a particular type of Destination to be created:

export const createDestination = async <
  T extends keyof DestinationTypeSettings,
  S = DestinationTypeSettings[T]
>(
  destinationType: T,
  settings: S
) => {}

Get in contact

If you have comments, questions or better ways to do anything that I have discussed in this post then please get in contact.