Visual analysis

Let the agent watch your footage — AI video understanding finds best ranges, hooks, and quality risks before it cuts.

Before the agent cuts anything, it can actually watch your footage. Visual analysis runs AI video understanding over your media and caches an editorial read — best ranges, hook candidates, quality warnings, crop guidance — so every later tool (highlights, reframe, B-roll) decides from what's on screen instead of guessing. Run it once per batch of footage; the results persist and get reused.

Copy-paste prompts

"Watch the footage and tell me what you see before editing."

"Analyze all imported clips, then pick the strongest 30 seconds."

"Sample visual context on the selected clip and flag any quality problems."

"Re-analyze the footage — I replaced the main clip."

You rarely need to ask for it explicitly: prompts like "make a Reel from this" trigger analysis automatically as the agent's first look at the footage.

What the agent actually does

When the agent analyzes your footage, it:

For videos, it tries Pegasus (TwelveLabs) full-video understanding first — an AI model that watches the whole clip, not isolated frames.
If full-video understanding isn't available for a clip, it falls back to sparse frame labeling with Claude vision (a handful of frames per asset, labeled for content).
Results are cached in the project and appear in every agent snapshot, so follow-up tools and later chat turns reuse them without re-analyzing.

Each analyzed asset can include:

Field	What it tells the agent
`sceneSummary`	What's happening on screen
`editorialSummary`	How to use the footage in an edit
`bestRanges`	Ranked timestamp ranges worth keeping — feeds highlight extraction
`suggestedHooks`	Hook lines grounded in the actual content
`qualityWarnings`	Shaky, dark, blurry, or otherwise risky sections
`captionRisks`	Regions where captions would collide with on-screen content
`cropGuidance`	Where to bias the crop for vertical reframing
`onScreenText`	Text rendered in the footage (so crops and overlays avoid it)
`tags`	Content tags: face, product, screen, and similar
`hasFaces`	Whether people are on screen

Each result also records its source: pegasus (full-video), frames (frame labels), or mixed.

Parameters

Parameter	Default	What it controls
`mode`	`auto`	What to analyze: `auto` (selected clips, else timeline media), `selection`, `timeline`, or `all` imported footage
`maxFrames`	24	Global frame budget for a fallback frame-sampling pass
`maxFramesPerAsset`	8	Frame cap per video in fallback mode — long videos are capped here
`forceRefresh`	false	Re-analyze even when a cached result exists

Say "analyze all imported footage" to widen the scope, or "re-analyze" to force a refresh after you swap media.

How other tools use it

Extract best moments keeps Pegasus-ranked bestRanges when they exist, instead of relying on audio energy alone.
Smart reframe uses cropGuidance, tags, and onScreenText to bias the vertical crop — faces stay in frame, rendered text doesn't get sliced.
B-roll insert picks bin assets whose analyzed content matches the spoken moment.
Hooks quote suggestedHooks or the transcript rather than inventing stock phrases.

When to do it manually

There's no manual equivalent — this is agent-only. The manual path is your own eyes: scrub the preview and set markers on the moments you want to keep.

Limits

Metered on the free plan. Full-video analysis counts against your monthly AI-video-analysis allowance — see Plans and limits.
Fallback is sparse. Frame labeling looks at up to 8 frames per video, so brief moments between samples can be missed.
Cached, not live. Results reflect the footage at analysis time; replace a clip and ask the agent to re-analyze (forceRefresh).
Best-effort labels. Summaries and tags are usually right but not guaranteed — sanity-check edits built on them.
If no analysis model is configured, the tool falls back to file metadata only.