Where your video lives has always mattered, it just matters a hell of a lot more now.
When you add video to your codebase, it’s not just something people watch anymore. That’s often one of the big workflows you’ll need to handle, but it’s certainly not the only one anymore.
Jon, my co-founder and fearless leader, recently wrote about video's transition from content, to features, and now data. I want to get into what that means practically.
Spoiler: this totally unbiased author thinks a video file on Mux is worth more than anywhere else. That was true before AI came onto the scene. It's more true now.
The mistake: ship and move on
The most common mistake I’ve seen from startups that touch video is treating it as something to ship and move on from. There’s a commodity mental model tied up in there: video infrastructure is just a pipe. Encode, cache, serve. The end.
That model puts video in a box with all your other static assets. Historically it’s also meant that really important decisions never get revisited (”why did you choose this quality ladder for your video?” “oh…no one knows, the person that wrote that left 3 years ago”). These days it ignores the boatload of context and data that you could use to build better, or entirely different, products around video.
If all you need is a pipe, you can find some pretty cheap pipes. On top of probably having a pretty mediocre video experience (which might be a reasonable tradeoff depending on what you’re building!), you’d be leaving a ton of value locked away inside those video files.
If a picture is worth 1000 words…
Then most videos are worth ~30k words per second. I don’t make the rules, that’s the math.
The frames themselves are where a lot of products are gravitating right now. There are a lot of LLM and ML tools that are good at processing images, so just run each frame of video through that sucker. Easy, peasy lemon squeezy.
Except…why? That’s absolutely the right call if you’re doing something like tracking people on a CCTV camera, but understanding what you actually need from your content allows you to get a lot more creative (and efficient). The combination of audio and the resulting transcripts, visual structure of scene boundaries, storyboards, and shots, and behavioral data of how viewers are actually engaging with the content gives you a wild amount of power.
Mux was built for this moment (but not because of it)
I wish I could say that we saw the future and spent 8 years building a product tailor made for real-world AI workflows around video…but we built video for people. Turns out that a lot of the things we spent a lot of time working on, from features to documentation, also translate well to this new world we find ourselves in.
We started with Mux Data because we believed great video infra needs great observability. We built magic thumbnails to make it easier to get preview images from any part of the video. Transcripts for accessibility. Storyboards for thumbnail scrubbers. The list goes on and on, but importantly, we built all of these as developer-first APIs so we could be programmable infrastructure, not a black box.
All of those things made it easier to build great video products. The same features we built for people help robots work with our video too. Now all of these features help turn streaming video into primitives and context.
Delivery doesn’t have to be the point anymore
Up to this point in Mux’s history, “video infrastructure” ultimately meant “get video to eyes.” Sure, some use cases involved a lot less delivery than others, but generally that was the primary thing people wanted us to do well. Now there are entire use cases that involve getting video in, understanding it, acting on it, and maybe delivering it.
Moderation, intelligence, classification, content understanding…none of those require a viewer on the other side to have value. Selfishly, I feel blessed to not have to write a blog post about our big pivot: we’re still video infrastructure, but today that infrastructure inherently includes intelligence the same way it inherently includes things like encoding.
Not knowing what you’ll build is the best feature
The reason I fell in love with developer tools was because of the possibilities they unlock. With the right set of tools and creativity, you can build whatever you can imagine, and you can now do it faster and more reliably.
As a dev tool company, the dream is not to know what people will build. It’s incredible to finally feel like it’s truly open ended on our infrastructure: it’s not just what you’re able to do with your video today, it’s what you’ll be able to do with it.
That’s why where video lives matters.
