Use the @mux/ai library to automatically generate titles, descriptions, and tags for your videos via LLMs
Automatically generating video metadata like titles, descriptions, and tags helps you build better search experiences, improve content discovery, and save time on manual content curation. The @mux/ai library makes this straightforward by analyzing video transcripts and storyboard images to generate metadata.
Before starting, make sure you have:
npm install @mux/aiSet your environment variables:
# Required
MUX_TOKEN_ID=your_mux_token_id
MUX_TOKEN_SECRET=your_mux_token_secret
# You only need the API key for the provider you're using
OPENAI_API_KEY=your_openai_api_key # OR
ANTHROPIC_API_KEY=your_anthropic_api_key # OR
GOOGLE_GENERATIVE_AI_API_KEY=your_google_api_keyimport { getSummaryAndTags } from "@mux/ai/workflows";
const result = await getSummaryAndTags("your-mux-asset-id", {
tone: "professional" // or "normal" or "sassy"
});
console.log(result.title);
// "How to Build a Video Platform in 2025"
console.log(result.description);
// "Learn the fundamentals of building a modern video platform..."
console.log(result.tags);
// ["video streaming", "web development", "tutorial", "javascript"]You can control the style of generated content with the tone option:
// Professional tone - formal and business-appropriate
const professional = await getSummaryAndTags("your-mux-asset-id", {
tone: "professional"
});
// Normal tone - balanced and conversational (default)
const normal = await getSummaryAndTags("your-mux-asset-id", {
tone: "normal"
});
// Sassy tone - playful and engaging
const sassy = await getSummaryAndTags("your-mux-asset-id", {
tone: "sassy"
});Here's some example titles for each tone, based on the same demo video of Mux's thumbnail API:
@mux/ai supports three AI providers:
gpt-5-mini model - Fast and cost-effectiveclaude-sonnet-4-5 model - Great for nuanced understandinggemini-2.5-flash model - Balance of speed and qualityconst result = await getSummaryAndTags("your-mux-asset-id", {
provider: "anthropic", // or "openai" or "google"
model: "claude-opus-4-5" // Optional: override default model
});By default, @mux/ai analyzes both the storyboard images and transcript. Storyboard images are always included, but you can optionally exclude the transcript:
// Exclude transcript (faster, uses only visual analysis)
const result = await getSummaryAndTags("your-mux-asset-id", {
includeTranscript: false
});You can override specific parts of the prompt to tune the output:
const result = await getSummaryAndTags("your-mux-asset-id", {
promptOverrides: {
system: "You are a video content specialist focused on technical tutorials.",
instructions: "Create a title under 60 characters and exactly 5 tags focused on technical concepts."
}
});For automated metadata generation when videos are uploaded, you should trigger the call to get the summary and tags from the video.asset.track.ready webhook:
export async function handleWebhook(req, res) {
const event = req.body;
if (event.type === 'video.asset.track.ready' &&
event.data.type === 'text' &&
event.data.language_code === 'en') {
const result = await getSummaryAndTags(event.data.asset_id, { tone: "professional" });
await db.updateVideo(event.data.asset_id, { title: result.title, description: result.description, tags: result.tags });
}
}Once you have automatically generated metadata, you can:
Under the hood, @mux/ai handles:
gpt-5-mini is cost-effective for most use cases