Service workers are underrated, and building media proxies proves it

Service workers have gotten unfairly typecast in the web development world. They're often just seen as a way to do fancy caching in your progressive web app, and most service worker guides primarily discuss your path to offline mode as an advanced use case. Your standard service worker proxy example looks something like this:

self.addEventListener("fetch", (event) => {
  // Basic caching example from MDN or something.
});

Here's the thing, though: service workers are an HTTP proxy. Yes, HTTP proxies are often used for...well, yeah, caching. But they can also go so much further, particularly for media. A proxy can manipulate the request itself — like adding headers to a request before passing it through to the origin, for example. It can resize or convert images. It can add a layer of security to an HLS stream, or stitch in an ad.

It’s time to explore more creative ways to use service workers.

OK, hold up; what is a service worker, anyway?

A service worker is first-party JavaScript that can intercept requests for all of the pages in the scope it's installed on. This first-party element is critical; for security reasons (you know, the whole intercept-all-HTTP-traffic thing), you cannot install a service worker that's hosted on a different origin.

For example, if you want to install a service worker on your website, https://creedthoughts.video, and have it intercept traffic for any page on that domain, you need to host the service worker at the root of the site: https://creedthoughts.video/service-worker.js.

If you install it at https://creedthoughts.video/dreams/service-worker.js, it will only work for pages nested under /dreams, such as https://creedthoughts.video/dreams/page/foo.

Once a service worker is installed, you can set up an event listener for fetch requests. You can manipulate request events and respond differently, or just cache along the way, or make other requests before passing it along.

Are these like web workers?

An important aspect of both service workers and web workers is that they run in their own context. If you were to do something really computationally expensive in your web page's normal JavaScript, you'd start to interfere with other code having its turn to run. In the media world, when you frequently manipulate media (such as transmux video to improve browser compatibility with modern streaming formats), you don't want to do all that work in your main thread.

Web workers allow you to offload that work to a separate JavaScript context from your interface, which means you can write code that does a lot of intensive work without interfering with the web page's interactivity. Both web workers and service workers don't provide direct access to the DOM, so communication between worker code and your main page needs to be done via messages.

Well that sounds neat, but is this just one of those Chrome things?

Service workers are well supported across evergreen browsers, including mobile!

Let’s use service workers to manipulate media

An overlooked element of modern streaming video: It's a lot of text files. At Mux, we primarily use a format called HTTP live streaming, or HLS. If you’ve ever looked at a WinAmp playlist, HLS will probably look pretty similar. We've written about this in depth, but the tl;dr is that HLS is made up of manifests that ultimately list out chunks of video in order. Here’s how it works:

• First, the player downloads the multivariant playlist. This lists all the available versions of the video, with different resolutions, codecs, etc.
• The player then looks through this playlist, picks which version of the video it can support, and downloads that version’s playlist.
• Finally, the player starts loading each chunk (or segment) of video listed in the playlist. Note: if it's a live stream, it will keep requesting this playlist over and over, because the stream will append new segments to the end of the playlist.

The fact that HLS is ultimately just text files that contain lists of other text files or video segments means we can make some pretty interesting things happen simply by editing those text files. What a good use case for a proxy where we just rewrite some text before responding!

A note on the code in this post: It's (mostly) all written by The Machines (i.e., generated by ChatGPT). The goal was to focus on the ideas, so thanks to ChatGPT, we’ve got some functional, albeit pretty slow and ugly, examples.

The feature/support requests

At Mux, we build online video infrastructure. To put it simply: We take video in and put modern streaming out (and give you insights to your viewers’ Quality of Experience along the way), all via a single API. When everything works well, folks are generally really happy. But what happens when our abstraction or features aren't quite right?

I had a few conversations with customers recently where two specific asks came up:

Our adaptive bitrate streaming is too...adaptive

One of the features of modern video streaming is that it adapts to the user's device and network capabilities. A stream might include anything from a very low bitrate and 240p resolution stream all the way up to a beefy 4k stream. The player can pick which stream it can support depending on its bandwidth (among other factors). This allows the player to stream video smoothly with as little buffering as possible and only download as much content as it can support.

The downside is that sometimes the viewer would rather wait for the content to buffer because of the content itself. For one of our customers that displays screencasts in their player, the issue is that a slide with code on it isn't very readable at 240p. Our own Mux Player will have rendition selection built in, but that won't reliably work on all players or with native HLS support (iOS and Safari).

For this example, the thought was: What if we used a service worker to proxy requests for the multivariant playlist, then removed any rendition playlists that were below a specified resolution?

Setting up our service worker

One behavior of service workers is that they won't intercept requests on the first load. To get around this limitation, we'll have an index.html page that loads the service worker and will link to our player page from there. The index can register the service worker like this:

if ("serviceWorker" in navigator) {
  navigator.serviceWorker
    .register("/resolution-filtering/service-worker.js", {
      type: "module",
    })
    .then((registration) => {
      console.log("Service Worker registered with scope:", registration.scope);
    })
    .catch((error) => {
      console.log("Service Worker registration failed:", error);
    });
}

This registers a service worker that will work for pages under /resolution-filtering/ and log success or failure.

The service worker itself

This is the /resolution-filtering/service-worker.js script we loaded in /resolution-filtering/index.html above. In practice, we might want this to be more configurable, but to keep things simple we're just going to hardcode a few basic variables, like our minimum resolution.

The install and activate event listeners are for setup. In the fetch event handler, we check to make sure we're handling the multivariant playlist before we start manipulating things. For Mux, just checking that the playlist is coming from stream.mux.com works here, since our rendition playlists come from a different origin.

Once we know we're dealing with a multivariant playlist, we can override the response on the event (event.respondWith). Here's what we're going to do before sending the new response:

1. Go get the original playlist.
2. Split the playlist by newlines.
3. For each line that includes a resolution (`EXT-X-STREAM-INF`), check if it's greater than or equal to the `MIN_RESOLUTION`.
4. If it is, add that line and the next line (which is the URL) to the new array of filtered lines.
5. If the line contains a URL, it's a rendition, so ignore it since it’s already been handled by the previous step.
6. Otherwise, the line likely contains other HLS tags, so pass the line through as-is to make sure it’s copied over to our newly-edited playlist

const MULTIVARIANT_HOST = "stream.mux.com";
const MIN_RESOLUTION = 720; // Set the minimum resolution threshold

self.addEventListener("install", async function (event) {
  console.log("Installing service worker");
  await event.waitUntil(self.skipWaiting()); // Activate worker immediately
  console.log("Installed service worker");
});

self.addEventListener("activate", async function (event) {
  console.log("Activating service worker");
  await event.waitUntil(self.clients.claim()); // Become available to all pages
  console.log("Activated service worker");
});

self.addEventListener("fetch", (event) => {
  const url = new URL(event.request.url);

  if (url.hostname === MULTIVARIANT_HOST && url.pathname.endsWith(".m3u8")) {
    console.log("Fetching and filtering playlist", event.request.url);
    event.respondWith(fetchAndFilterPlaylist(event.request));
  }
});

async function fetchAndFilterPlaylist(request) {
  const response = await fetch(request);

  const text = await response.text();

  const filteredText = filterPlaylist(text);
  const filteredResponse = new Response(filteredText, {
    headers: response.headers,
  });

  return filteredResponse;
}

function filterPlaylist(text) {
  const lines = text.split("\n");
  const filteredLines = [];

  for (let i = 0; i < lines.length; i++) {
    const line = lines[i];

    if (line.startsWith("#EXT-X-STREAM-INF")) {
      const resolution = line.match(/RESOLUTION=(\d+)x(\d+)/);

      if (resolution && parseInt(resolution[2], 10) >= MIN_RESOLUTION) {
        filteredLines.push(line);
        filteredLines.push(lines[++i]);
      }
    } else if (line.startsWith("https://")) {
      // This should be a rendition playlist line, which should always
      // be handled by the above if statement so we should ignore it here.
    } else {
      // This makes sure the rest of the playlist is copied over.
      filteredLines.push(line);
    }
  }

  return filteredLines.join("\n");
}

There you go! Now you can nest a new page (or pages!) for your players under the /resolution-filtering directory/route. As long as you load that /resolution-filtering/index.html page where you register the service worker first, any of those player pages should now have filtered manifests coming back.

Seamlessly playing back a few videos

Another use case we hear from folks is that they want to play back a number of assets seamlessly, one after the other. Even if you get creative with preloading, reusing the same player and loading a new video into it is never going to be perfectly smooth. The other method is using a pool of players and switching between them, which is as fun to build and maintain as it sounds.

Another option is to "stitch" the manifests together so the player gets a single video. In the UI, you could show them as separate playlist items and then just set the video time or display chapter markers.

To do this, HLS has a special tag called a discontinuity, EXT-X-DISCONTINUITY, that can be inserted between segments that aren't actually related. The discontinuity flag basically resets the decoder so the player knows to expect something different for the next segment. This approach isn't without its own issues, but it should be fine as long as you're stitching together videos with the same general encoding profile.

The service worker

This time we'll work from a new directory, /stitching. We'll have basically the same index.html as the previous example, but instead we'll register /stitching/service-worker.js. The install and activate handlers are the same as well.

This time, we're going to specify a special hostname that our service worker will intercept. This hostname doesn't have to be real (but it can be; more on that later), since our service worker is never going to let anything hit that domain. We're going to rely on some Mux patterns around URLs and such, but otherwise this should work with any HLS manifests. Here's what we're going to do:

1. Our player URL is going to be https://${STITCH_HOST}/${PLAYBACK_ID_1}/${PLAYBACK_ID_2}/...
2. Our service worker intercepts the request and fetches each multivariant playlist in the URL (https://${STREAM_DOMAIN}/${playback_id_#}.m3u8).
3. To simplify things, we're going to fetch the highest-resolution entry from each multivariant playlist.
4. We'll keep the tags from the first manifest the same and then include all of the segments from each playlist with #EXT-X-DISCONTINUITY between them.
5. 5. We’ll slap an #EXT-X-ENDLIST on the end to let the player know it's done.

const CACHE_NAME = "stitched-cache-v1";
const STITCH_DOMAIN = "stitch-o-matic.mux.dev";
const STREAM_DOMAIN = "stream.mux.com";

self.addEventListener("install", async function (event) {
  console.log("Installing service worker");
  await event.waitUntil(self.skipWaiting()); // Activate worker immediately
  console.log("Installed service worker");
});

self.addEventListener("activate", async function (event) {
  console.log("Activating service worker");
  await event.waitUntil(self.clients.claim()); // Become available to all pages
  console.log("Activated service worker");
});

self.addEventListener("fetch", (event) => {
  const url = new URL(event.request.url);

  if (url.hostname === STITCH_DOMAIN) {
    console.log("Fetching and stitching manifests", event.request.url);
    event.respondWith(fetchAndStitchHighestResolutionManifests(url));
  }
});

async function fetchAndStitchHighestResolutionManifests(url) {
  const playbackIds = url.pathname.split("/").filter((id) => id);
  const manifests = await Promise.all(
    playbackIds.map((id) =>
      fetchHighestResolutionManifest(`https://${STREAM_DOMAIN}/${id}.m3u8`)
    )
  );

  const stitchedManifest = stitchManifests(manifests);
  const stitchedResponse = new Response(stitchedManifest, {
    headers: { "Content-Type": "application/vnd.apple.mpegurl" },
  });

  return stitchedResponse;
}

async function fetchHighestResolutionManifest(manifestUrl) {
  const response = await fetch(manifestUrl);
  const text = await response.text();
  const highestResolutionUrl = getHighestResolutionManifestUrl(
    text,
    manifestUrl
  );

  const highestResolutionManifestResponse = await fetch(highestResolutionUrl);
  const highestResolutionManifest =
    await highestResolutionManifestResponse.text();

  return highestResolutionManifest;
}

function getHighestResolutionManifestUrl(playlist, baseManifestUrl) {
  const lines = playlist.split("\n");
  let highestResolution = -1;
  let highestResolutionUrl = "";

  for (let i = 0; i < lines.length; i++) {
    const line = lines[i];
    if (line.startsWith("#EXT-X-STREAM-INF")) {
      const resolution = line.match(/RESOLUTION=(\d+)x(\d+)/);
      if (resolution) {
        const currentResolution = parseInt(resolution[2], 10);
        if (currentResolution > highestResolution) {
          highestResolution = currentResolution;
          highestResolutionUrl = new URL(lines[++i], baseManifestUrl).href;
        }
      }
    }
  }

  return highestResolutionUrl;
}

function stitchManifests(manifests) {
  const stitchedLines = [];

  for (let i = 0; i < manifests.length; i++) {
    const manifest = manifests[i];
    const isFirstManifest = i === 0;
    const lastManifest = i === manifests.length - 1;
    const lines = manifest.split("\n");

    for (let b = 0; b < lines.length; b++) {
      const line = lines[b];
      if (isFirstManifest) {
        if (
          line.startsWith("#EXTM3U") ||
          line.startsWith("#EXT-X-VERSION") ||
          line.startsWith("#EXT-X-PLAYLIST-TYPE") ||
          line.startsWith("#EXT-X-TARGETDURATION")
        ) {
          stitchedLines.push(line);
        }
      }

      if (line.startsWith("#EXTINF")) {
        stitchedLines.push(line);
        stitchedLines.push(lines[b + 1]);
      }
    }

    if (!lastManifest) {
      stitchedLines.push("#EXT-X-DISCONTINUITY");
    }
  }

  stitchedLines.push("#EXT-X-ENDLIST");
  return stitchedLines.join("\n");
}

There we go again! Now we can use our new fake hostname to create stitched playlists and get that sweet, sweet seamless playback.

Bonus: you can deploy service workers to the edge

Thanks to edge runtimes such as Cloudflare Workers using the fetch API, both of these examples will magically Just Work™ if you deploy them there. Without getting into all the wrangler details, I just went and created a worker, copied and pasted our stitching example into the quick editor, deployed it, and got a URL that looks like this: red-hall-71db.mux-devex7423.workers.dev

You can try things out by stitching any playback IDs together. For example: https://red-hall-71db.mux-devex7423.workers.dev/${EXAMPLE_1}/${EXAMPLE_2}

If you're in an environment where the service worker works, great! You don't have to pay for any edge computing, because the browser is doing all the lifting. But you can also use that same URL in something like your native app or on another host, and it will still work just fine because the edge worker is handling it.

The world needs more service workers

HLS manifest editing is just one application where service workers shine, but there's gotta be so many other possibilities to get creative and use them in ways that we haven't even thought of yet. It all starts with folks like you reading through some educational resources and applying the tech to the industry you know best.