The Mux Robots API uses AI to extract structured data from your videos. Instead of integrating third-party AI services, managing async job queues, handling retries and failures, and writing custom glue code to tie it all together, you can send a single API request with an asset ID and get results back directly.

The Robots API is our recommended, batteries-included approach for adding AI to your video workflows — it handles model selection, prompting, retries, and webhook delivery so you can focus on using the results.

Robots is priced per Mux AI unit consumed by each job. The first 100,000 units each month are free. See Mux Robots pricing for rates.

Workflows that analyze, generate, or translate language share a set of verified languages and language tags. Languages outside that set are processed on a best-effort basis.

Choose an AI approach

Start with the result you need and how much of the pipeline you want to own:

For production apps, prefer Robots when a built-in workflow meets the need. Use its documented parameters and output_steering controls before building a custom pipeline. Frame analysis is available in both Robots and @mux/ai; visual input alone does not require choosing one over the other. A transcript-only call cannot establish what is visible in the video.

All three approaches may involve Mux asset/storage costs. Robots jobs consume Mux AI units; custom pipelines use your own AI-provider billing. See pricing and the chosen workflow's prerequisites before running a batch.

What you can do

Most workflows accept output_steering to guide style, audience, terminology, and which part of the video is analyzed. See each workflow's guide for its options.

Workflow prerequisites

Use a ready asset in the same environment as your token. It must no longer be live and must have a public or signed playback ID; DRM-only playback IDs are not supported. Pass the asset ID to the job, even when the workflow reads media through a playback ID.

Required resources must be ready before a standalone job is created. Recommended resources improve output, but are not mandatory unless the table identifies a specific input or mode that requires them.

This table covers the public v1 workflows. Check each workflow's guide and API reference for duration limits, supported languages, and parameter-specific restrictions.

Prepare and wait for captions

For new uploads, enable auto-generated captions in the asset inputs. For an existing asset, request captions or upload subtitles, or run Generate premium captions.

Retrieve the asset and wait for the source text track to have status: "ready". In v1, caption sources are subtitle tracks (type: "text", text_type: "subtitles"). An audio-only asset may also use its single ready legacy text transcript without subtitle metadata. A chapter track is not a subtitle source. Asset readiness and caption readiness are separate: video.asset.ready does not guarantee the track is ready. Use video.asset.track.ready or bounded polling of the asset's tracks.

When language_code selects a caption source, it must match the ready track's language exactly, including case and region: en does not match en-US. Without it, automatic selection uses the first ready subtitle in asset track order. Edit and translate captions instead identify the source with track_id. Retrieve that ID from the asset; do not use a job ID or a playback ID.

If a source track is still preparing, wait. If it failed, is missing, or is in the wrong language, repair or choose the source before retrying the same job. A successful premium-caption job with upload_to_mux: false supplies a temporary subtitle URL, not a track that downstream workflows can select.

Let a Directive prepare dependencies

For recurring work, declare prerequisites as resources and reference them in each binding's inputs. This v1 configuration generates subtitles when needed and waits for them before chapter generation:

{
  "name": "Prepare subtitles and generate chapters",
  "subject": { "type": "video.asset" },
  "resources": [
    {
      "reference_id": "captions",
      "type": "video.asset.track",
      "kind": "subtitles",
      "source": { "via": "mux_api", "action": "generate_subtitles" }
    }
  ],
  "bindings": [
    {
      "reference_id": "chapters",
      "workflow": "generate-chapters",
      "inputs": ["captions"],
      "parameters": {}
    }
  ]
}

Create the Directive with POST /robots/v1/directives, then attach it to new assets or start a run for an existing asset as described in the Directives guide. A subtitle-generating action still needs an audio source. Inspect the run's node states if a resource or producer fails; declaring the dependency does not guarantee it can be produced. Shot generation may incur separate Mux Video charges.

How it works

The Robots API is organized around jobs. A job represents a single unit of work — one row from the table above. Each type of work is called a workflow.

The general pattern is:

1. Create a job by sending a POST request to the workflow endpoint with your Mux asset ID and any configuration parameters.
2. Wait for the job to complete. Jobs move through statuses: pending → processing → completed (or errored). A job can also be cancelled via the cancel endpoint. We strongly encourage using webhooks for status changes, but jobs can also be retrieved through the GET /robots/v1/jobs/{workflow}/{JOB_ID} endpoint.
3. Read the results from the job's outputs field once the status is completed.

Webhooks

Every status change fires a webhook event. This lets you react to completed jobs without polling. The event type follows the pattern robots.v1.job.{workflow}.{status}:

For example, a completed moderation job sends a robots.v1.job.moderate.completed event. A completed summarization job sends robots.v1.job.summarize.completed.

Note that multi-word workflow names use underscores in event types (e.g. robots.v1.job.ask_questions.completed, robots.v1.job.find_key_moments.completed), even though the API endpoints use hyphens.

The webhook payload's data field contains the full job object — the same shape you'd get from polling the job endpoint — so you can read outputs directly from the event without making an additional API call.

All Robots API endpoints live under /robots/v1/.

Authentication

The Robots API uses the same Mux API token authentication as the rest of the Mux API. Include your token ID and secret using HTTP Basic Auth.

curl https://api.mux.com/robots/v1/jobs/summarize \
  -H "Content-Type: application/json" \
  -X POST \
  -d '{ "parameters": { "asset_id": "YOUR_ASSET_ID" } }' \
  -u ${MUX_TOKEN_ID}:${MUX_TOKEN_SECRET}

Job lifecycle

Every job includes the following common fields:

Managing jobs

You can retrieve, list, and cancel jobs across all workflows.

Retrieve a job by workflow and job ID:

curl https://api.mux.com/robots/v1/jobs/{workflow}/{JOB_ID} \
  -u ${MUX_TOKEN_ID}:${MUX_TOKEN_SECRET}

Replace {workflow} with the workflow name used when the job was created (e.g. summarize, moderate, generate-chapters). The response includes the full job object with outputs when the job is completed.

List jobs with optional filters:

curl "https://api.mux.com/robots/v1/jobs?workflow=summarize&status=completed&limit=10" \
  -u ${MUX_TOKEN_ID}:${MUX_TOKEN_SECRET}

You can filter by workflow, status, asset_id, and paginate with limit (default 25, max 100) and page (default 1). Each entry is a summary with id, workflow, status, created_at, updated_at, and _links.self; follow _links.self.href to retrieve the full job.

Cancel a job that is pending or processing. The job's status will transition to cancelled:

curl -X POST https://api.mux.com/robots/v1/jobs/{JOB_ID}/cancel \
  -u ${MUX_TOKEN_ID}:${MUX_TOKEN_SECRET}