Guau! Wauw! Vau! Wow! VOD auto-generated captions are now available in 22 languages!

Back in October last year, Mike announced that we were adding support for automatically generating captions for on-demand video content. Since then, we've seen hundreds of customers adopt auto-generated captions across dozens of different types of use cases, including fitness apps, events platforms, product reviews, and even fishing tutorials!

Today we're making 3 updates to the VOD auto-generated captions feature that we're really excited about:

1) More languages, fewer headaches

First, we've added 21 more languages to the feature. Now you can increase accessibility for content created all over the world. These languages are broken up into two sets.

The first set is French, German, Italian, Portuguese, and Spanish. These languages join English in our stable languages list. The underlying Whisper models that power these languages are mature and have a high level of accuracy for content with a good audio input.

We've also added support for Bulgarian, Catalan, Croatian, Czech, Danish, Dutch, Finnish, Greek, Norwegian, Polish, Romanian, Russian, Slovak, Swedish, Turkish, and Ukrainian. These languages will remain in beta as we iterate and better understand the real-world accuracy of the underlying models.

You can check out the full list of languages, and their relative stability in the VOD auto-generated captions documentation. Desperate to use a language that you don't see on this list? We'd love to hear which languages we should be investing in next!

2) An API for retroactively adding captions

Second, we've added a new API, which allows you to retroactively add generated captions to your existing content. This means you can add captions to your content after you create your asset. You can use this API for up to 7 days after you create an asset.

You can retroactively add captions to any smart asset by POSTing to the generate-subtitles endpoint on the asset audio track that you want to generate captions for, as shown below:

// POST /video/v1/assets/${ASSET_ID}/tracks/${AUDIO_TRACK_ID}/generate-subtitles

{
    "generated_subtitles": [
        {
            "language_code": "en",
            "name": "English (generated)"
        }
    ]
}

Here are a few places where you might find this API useful:

1. If your CMS integration makes it hard to change the underlying API calls you make to Mux, you can listen to video.asset.ready webhook instead, and then immediately request captions are generated
2. If you want to retroactively add (or replace) captions on an asset that was generated from a live stream
3. If you have an asynchronous process to run before requesting captions, for example, to allow the user to identify the content language
4. Or… you simply forgot to ask us to generate captions (whoops, it happens to all of us!)

If you need to use this API to backfill captions to assets created longer than 7 days ago, please reach out and we'd be happy to help. Please note that there may be a charge for backfilling captions onto large libraries.

3) It's GA, go wild!

Finally, and most importantly, the VOD auto-generated captions feature is now GA! This means that the APIs are now stable, and the product is ready for scale 📈.

So what are you waiting for? Get captioning!