Generate a title, summary, tags, chapters, scenes, and key moments from a video or audio asset with a single call to the Mux Robots API, and steer the time range, scene and chapter density, audience, and topic vocabulary.
Generate a title, summary, tags, chapters, scenes, and key moments together from one Mux asset or a selected time range. Use one job to fill metadata in your CMS, add chapter navigation to a player, and find highlights to review for clips. All outputs are built from shared transcript and visual evidence. See the Analyze Structure API referenceAPI for the full endpoint specification. See Mux Robots pricing for unit costs.
If you previously used Find Scenes for segmentation, read outputs.scenes to get scene data alongside the other metadata. Use Directives to run analysis automatically as assets are ingested.
The asset must be ready, no longer live, and have a public or signed playback ID. Video assets can be analyzed without captions, but captions provide spoken context. Audio-only assets require a ready caption track. You can generate captions or add subtitles before running the workflow. For video, the workflow requests or waits for Mux Shots; shot generation can incur separate Mux Video charges.
analyze-structure jobcurl https://api.mux.com/robots/v1/jobs/analyze-structure \
-H "Content-Type: application/json" \
-X POST \
-d '{
"parameters": {
"asset_id": "YOUR_ASSET_ID",
"output_language_code": "en"
}
}' \
-u ${MUX_TOKEN_ID}:${MUX_TOKEN_SECRET}This request is asynchronous. The POST returns immediately with the job in pending status and does not include results. We strongly recommend listening for the robots.v1.job.analyze_structure.completed webhook. The payload contains the full completed job, so no follow-up API call is needed. If webhooks aren't an option, poll GET /robots/v1/jobs/analyze-structure/{JOB_ID} with the id from the response until the status is completed. Jobs are automatically deleted after 30 days.
| Parameter | Type | Description |
|---|---|---|
asset_id | string | Required. The Mux video or audio asset ID to analyze. |
language_code | string | Preferred caption language as a BCP 47 tag. An explicit value must match a ready source exactly, including case: en does not match en-US. When omitted, the first ready subtitle track is selected. Audio-only assets can also use a single ready legacy text transcript. |
output_language_code | string | BCP 47 language code for generated text. Auto-detected from available evidence when omitted. Transcript cues and quotes remain in their source language. |
output_steering | object | Controls for the analyzed time range, audience, terminology, topics, and scene and chapter density. See Output steering. |
update_asset_meta | boolean | When true, writes the generated title to asset.meta.title. Defaults to false. See Update the asset. |
update_asset_chapters | boolean | When true, replaces or creates a chapters track. Defaults to false. See Update the asset. |
See language support for Mux Robots for verified languages and language tags.
Scope bounds which media is analyzed. The other controls provide best-effort guidance and do not guarantee exact output.
| Field | Type | Description |
|---|---|---|
scope | object | Execution window in seconds on the original asset timeline. Supply start_time, end_time, or both. An omitted start means the beginning; an omitted end means the asset's end. The start must be nonnegative, the end must be positive, and the start must precede the end. An end past the asset's duration is clamped to the asset's end, and a range that starts at or after the asset's end is rejected. The start is inclusive and the end is exclusive. Returned timestamps remain relative to the original asset's start, rather than restarting at zero for the selected range. |
granularity | string | Moves scene and chapter density together. Supported values: coarse, balanced, and fine. |
audience | string | Intended audience, up to 160 characters. |
brand_terms | array of strings | Preferred terms to use when supported by the source. Supply 1–10 terms, up to 40 characters each and 240 characters combined. |
topic_taxonomy | object | Controlled vocabulary for topics. Optional name; required values array with 1–50 entries, each with a label and optional description and aliases. allow_other controls whether topics outside the vocabulary may be used. |
For example, analyze only the product demonstration between 30 and 150 seconds:
{
"parameters": {
"asset_id": "YOUR_ASSET_ID",
"output_steering": {
"scope": { "start_time": 30, "end_time": 150 },
"granularity": "fine",
"audience": "developers learning to upload videos",
"brand_terms": ["Mux Video"]
}
}
}Analysis returns metadata without writing it to the asset by default. Set update_asset_meta to true to write the generated title to asset.meta.title. Set update_asset_chapters to true to create a chapters text track, replacing existing chapters tracks.
{
"parameters": {
"asset_id": "YOUR_ASSET_ID",
"update_asset_meta": true,
"update_asset_chapters": true
}
}Each update reports its own outcome in outputs.asset_updates. An update failure does not fail successful analysis. If no chapters are generated, the chapters update is skipped, its outcome is omitted, and any existing chapters track is preserved.
Chapter replacement deletes existing chapters tracks before creating the new track. If deletion succeeds but creation fails, the asset can be left without chapters. Check asset_updates.chapters.status and replaced_track_ids even when the job is completed.
The outputs object is included once the job is completed, in the robots.v1.job.analyze_structure.completed webhook or the job's GET response. All timestamps are seconds on the original asset timeline, including fractional seconds.
| Field | Type | Description |
|---|---|---|
title | string | Generated title. |
summary | string | Generated summary. |
tags | array of strings | Generated tags. Empty when no tag qualifies. |
chapters | array | Ordered chapters with start_time, end_time, and title. Chapters are eligible when the analyzed range is at least 60 seconds and has at least two coverage segments, but may still be empty. |
scenes | array | Ordered video scenes with time boundaries, a title, and overlapping transcript cues. Always empty for audio-only assets. |
key_moments | array | Selected moments in timeline order, with time boundaries, a title, a quality score, and overlapping transcript cues. Empty when none qualify. |
asset_updates | object | Outcomes of requested asset updates. Title status is updated or failed; chapters status is created or failed, with track_id for the new track and replaced_track_ids for deleted tracks when present. |
Scenes and key moments can include audible and visual narratives and notable concepts. Scenes can also include a shot breakdown; key moments can include a verbatim quotable_segment with time and text. Transcript cues have start_time, end_time, and text, and remain in their source language.
See the API referenceAPI for the full nested output fields.
This example analyzes a two-minute video, consuming 4,000 units. It is the job delivered in the robots.v1.job.analyze_structure.completed webhook's data field and returned by GET /robots/v1/jobs/analyze-structure/{JOB_ID}:
{
"data": {
"id": "rjob_abc123",
"workflow": "analyze-structure",
"status": "completed",
"units_consumed": 4000,
"created_at": 1746403200,
"updated_at": 1746403260,
"parameters": {
"asset_id": "YOUR_ASSET_ID",
"output_language_code": "en"
},
"outputs": {
"title": "Uploading a video with Mux",
"summary": "A developer demonstrates creating a direct upload and playing the resulting video.",
"tags": ["video uploads", "video playback"],
"chapters": [
{ "start_time": 0, "end_time": 60, "title": "Create a direct upload" },
{ "start_time": 60, "end_time": 120, "title": "Play the uploaded video" }
],
"scenes": [
{
"start_time": 0,
"end_time": 60,
"title": "Create a direct upload",
"cues": [],
"visual_narrative": "A developer creates a direct upload in the dashboard.",
"shot_count": 2
},
{
"start_time": 60,
"end_time": 120,
"title": "Play the uploaded video",
"cues": [{ "start_time": 62, "end_time": 67, "text": "The video is ready to play." }],
"audible_narrative": "The developer explains that the upload is ready for playback.",
"visual_narrative": "A player loads the uploaded video.",
"shot_count": 2
}
],
"key_moments": [
{
"start_time": 60,
"end_time": 75,
"title": "First playback",
"overall_score": 0.85,
"cues": [{ "start_time": 62, "end_time": 67, "text": "The video is ready to play." }],
"quotable_segment": { "time": 62, "text": "The video is ready to play." }
}
]
}
}
}