Your archive staff spends 4 hours finding a 40-second clip. That number isn't hypothetical — it's what a regional broadcast operation told us when we first started talking to people in the media industry in 2023. They had 22 years of news footage, roughly 31,000 hours of video on LTO tape and spinning disk, and a search system that worked by matching against a spreadsheet of manually entered titles, rough timestamps, and subject codes written by whoever happened to be logging that day. When the original logger left, those code conventions left with them.
The video content problem isn't that there's too much video. It's that video search infrastructure hasn't meaningfully evolved in 20 years, while the volume of content has increased by orders of magnitude. You can find a document from 2003 in three seconds. Finding a specific moment in a 2003 news segment still requires someone to physically know where to look.
The Metadata Trap
Most video archives run on a media asset management system — a MAM — that stores structured metadata alongside the media files. MAM vendors have made these systems better at managing files, handling proxies, and integrating with NLE workflows. What they haven't solved is the fundamental problem: the search index is only as good as what a human typed into a field.
When a tape ingests, someone assigns a title, maybe a subject category from a controlled vocabulary, perhaps a few free-text keywords. That metadata describes the container, not the content. If a news segment covers a city council vote on zoning permits and the logger wrote "City Hall meeting — March 2018," you will never find that clip by searching for "zoning" or "permits" or "rezoning." The words that describe what actually happens in the video are locked inside the audio track and the on-screen graphics, completely invisible to any search query.
This isn't a failure of the MAM vendors. Structured metadata entry was a reasonable design choice when ingest volumes were measured in dozens of tapes per week. It breaks entirely when ingest volumes are measured in terabytes per day.
Why Transcription Alone Doesn't Fix It
The first answer most people reach for is speech-to-text. Run ASR across the archive, index the transcripts, problem solved. It's a genuine improvement — moving from zero searchable content to a full-text index of spoken audio is a meaningful step. But it leaves substantial signal on the table and creates new failure modes.
The obvious gap is anything that isn't speech. News chyrons — the lower-third text overlays — often contain information that isn't spoken aloud. A segment might show footage of a building with an on-screen text identifying the location while the narrator says something entirely different. The name of the building, the date stamp in the corner, the title card for an interview subject: none of that appears in an ASR transcript. For legal proceedings, those visual identifiers are often exactly what someone needs to find.
Less obvious is the ASR quality problem. Word error rate on broadcast audio with clean studio conditions is low, often under 5% with modern models. In the field, it degrades fast. Background noise, multiple speakers without proper diarization, non-native accents, domain-specific terminology — WER climbs toward 15–25% in realistic conditions, and errors cluster around proper nouns and technical terms, which are exactly the words you're most likely to search for. If a journalist's name is consistently mis-transcribed, every search for that journalist's coverage returns nothing.
We're not saying ASR is useless — we use it ourselves and it's a foundational layer. We're saying it's not sufficient on its own, and treating it as a complete solution leads to a search system that looks complete but has significant holes.
The Scene Understanding Gap
The third layer of missing information is visual content. Video is a visual medium. A significant fraction of what makes a clip useful or relevant is what appears on screen, not what's said. The specific camera angle, the visual composition, whether a public figure is in frame, what signage is visible, the visual context of a location — these are discoverable by a human watching the clip, and completely invisible to a text-based search index.
Frame-level image analysis has improved dramatically. Running visual classification across keyframes extracted from a video can identify objects, faces, settings, and scenes at reasonable accuracy levels. But naive keyframe extraction misses transitions, and keyframe-level indexing loses temporal context. Knowing that a face appears in frame 1,240 of a video isn't the same as knowing that the person is speaking during a specific segment, or that they appear in a specific visual relationship to another element in the shot.
The gap is that visual indexing has largely been treated as a separate problem from audio indexing, resulting in disconnected indices that can't answer queries that span both modalities. "Find the segment where [person] is speaking near [location]" requires correlating a face detection result with a diarized speaker segment — two separate pipelines whose outputs need to be joined at query time. Most implementations don't do this, so the query fails.
What the Actual Architecture Requires
Getting video search right requires treating the video as a unified multimodal document, not as a media container plus a text transcript plus an image gallery. The index needs to capture speech content (with speaker attribution from diarization), on-screen text (OCR applied to frames, not just keyframes), visual semantics (scene and object classification at shot level), and the temporal relationships between all of these signals.
Then the search layer needs to operate across all of those modalities simultaneously, ranking results by relevance across the combined signal — not running separate queries against separate indices and merging results. That distinction matters because the best clips for many queries are the ones where the visual and audio signals converge, not just the ones where one signal happens to match.
This is what we built Meshora to do. Not because the individual components are novel — ASR, OCR, visual classification are all established — but because nobody had assembled them into a coherent, query-time-unified index designed specifically for large video archives. The MAM vendors aren't going to do it; it's outside their product remit. The general-purpose search vendors don't understand the media-specific requirements. The gap is real, and the organizations sitting on 10,000-hour archives are feeling it every time their archive team spends an afternoon looking for a 90-second clip.
Who This Doesn't Help (Yet)
A few honest constraints worth naming. First, if your archive exists entirely on physical media — LTO tapes, Betacam, older formats — search indexing requires digitization, and digitization has to come first. We can index ProRes masters, MPEG-2 files, MP4s, and most common container formats, but we can't index what isn't digital. If your archive has a significant physical component, the digitization workflow is a prerequisite, not something Meshora can shortcut.
Second, archives with inconsistent timecode are a real challenge. Timecode discontinuities, broken sequences, and missing timecode on older materials create alignment problems when you're trying to link a search result to a specific moment in a file. We handle this better than most approaches, but it adds complexity and occasionally produces less precise hit points.
Third, if your use case requires near-real-time indexing of live ingest, current indexing latency is measured in minutes, not seconds. For a deep archive query workload, that's fine. For a newsroom that needs live content searchable within 30 seconds of ingest, it isn't.
The 20-year-old problem of video search being structurally broken isn't going to be fixed by adding more metadata fields to a MAM. The signal is in the content. Reaching it requires building an index that can see what the video contains — not just what a human typed about it.
