Gemini's Agentic Video Understanding Cuts Token Costs by 88% While Raising Accuracy

Google DeepMind has introduced agentic video understanding across Gemini 3.7 Flash, Gemini 3.6 Flash, and Gemini 3.5 Flash-Lite, changing how the models process video by letting them search and resample footage instead of scanning it uniformly frame by frame.

A different question than 'how many frames should I sample'

Before this release, Gemini's default video handling sampled frames at a fixed rate of one per second, an approach Google DeepMind calls static processing. It's a reasonable baseline, and it's still there, adjustable through the API if you want more or fewer frames per second. What agentic video understanding adds is a second mode: instead of committing to a sampling rate up front, the model searches across visual frames, audio, and transcripts to find the segments that actually matter, then resamples those specific time windows at higher frame rates when it needs finer detail, for example to catch an anomaly. The result, according to Google DeepMind, is up to 88% less token consumption, up to 66% lower cost, and up to 7% better accuracy on video-analysis tasks, three numbers that don't usually move in the same direction at once.

What the agent can actually find in a video

The headline capability is sub-second moment retrieval across multi-hour videos, meaning the model can locate a specific instant within footage that runs for hours without needing to have processed every second of it at full density. The same underlying search also supports counting: the feature can count repeated physical movements and distinct objects over time, which is a different task from simple retrieval since it requires the model to keep track of occurrences across a timeline rather than just pointing to one moment. None of this depends on a new video format or a separate upload path — the feature works with both direct video uploads and YouTube videos submitted through the Gemini API, which keeps the integration surface the same as before.

Turning it on is a single configuration change

Developers enable agentic video understanding by setting processing to agentic in the API configuration, rather than leaving it on the static default. There's no separate feature fee attached to this — Google DeepMind states the feature uses standard Gemini API token pricing, so the cost savings come from the reduction in tokens actually consumed, not from a discounted rate. The feature is live now in Google AI Studio and on the Gemini Enterprise Agent Platform, which means it's already reachable by anyone building against the Gemini API today rather than being locked behind a private preview.

Where this is headed next

Google DeepMind has said agentic video understanding is planned to roll out more broadly to Gemini app users across the Flash and Flash-Lite model lines, extending it beyond API developers to the consumer-facing app. It's also planned to power YouTube's Ask YouTube feature on video watch pages, which would put this exact search-and-resample mechanism behind a feature that ordinary viewers interact with directly, not just developers building agents. For anyone currently working with video inputs through the Gemini API, the practical next step is straightforward: switch the processing parameter to agentic in your existing configuration and compare token counts and accuracy against your current static-rate setup, since both are visible in the same billing and output structure you already use.

Specs

• Supported models: Gemini 3.7 Flash; Gemini 3.6 Flash; Gemini 3.5 Flash-Lite
• Maximum token reduction: 88%
• Maximum cost reduction: 66%
• Maximum accuracy improvement: 7%
• Default static processing rate: 1 FPS
• API processing mode: agentic
• Additional feature fee: None

Sources

• Google DeepMind — Google DeepMind