There's a version of this project that exists only in product demos: you point the system at an archive, it runs overnight, and everything is searchable by morning. That's not what happens when you're dealing with 40,000 hours of broadcast footage accumulated across 18 years, stored across multiple formats, with varying degrees of organization and a timecode history that looks like someone used a random number generator.
This is an account of what we actually encountered building an index for a corpus at that scale. Not every archive will have the same problems, but the failure modes we hit are representative of what you'll find in any long-running broadcast operation that wasn't originally designed with searchability in mind.
The Inventory Problem Comes First
Before any indexing can happen, you need to know what you have. This sounds obvious but is genuinely underestimated. In this archive, the file inventory was spread across a MAM system that tracked current operational files, three generations of deprecated file servers with inconsistent naming conventions, and a spreadsheet maintained by a former staff member that described tapes which had been partially digitized but not fully ingested into the MAM.
The first two weeks were spent writing a crawler to build a unified manifest: file path, container format, codec, duration, creation date, presence/absence of timecode track, and whether the file existed in the MAM. The result was 147,000 files totaling approximately 41,200 hours across ProRes 422, MPEG-2, H.264, and some legacy DV footage. About 12% of those files had no corresponding MAM record — they existed on disk but were invisible to the archive team's search tools.
The inventory step is not a Meshora-specific requirement. Any indexing pipeline starts here. If you don't know what you have, you can't systematically process it, and you'll end up with gaps in coverage that only become apparent when a search fails for footage you know should exist.
Format Heterogeneity and Transcoding Overhead
The archive was not a single format. The ProRes 422 files — roughly 60% of the corpus by duration — were the most recent material and processed cleanly. The MPEG-2 files, mostly from 2005–2014 capture, needed container normalization before ASR and visual pipelines could operate on them without frame-level artifacts. The DV files (a small fraction, roughly 3% by duration, covering the earliest content) were the most challenging: interlaced video, field-order inconsistencies, and some with significant audio dropout from aging tape stock.
We made an explicit decision not to transcode the source files. Transcoding 40,000 hours of video to a normalized format takes significant compute time and creates a storage duplication problem. Instead, we built format-adaptive ingestion paths that handle each source format in place, extracting audio and frame samples in format-appropriate ways. For MPEG-2 files, this meant using GOP-boundary-aware extraction to avoid mid-frame decode artifacts. For DV content, we applied field deinterlacing during frame extraction and compensated for audio channel dropout in the ASR preprocessing step.
This approach is more complex to implement but avoids a common trap: teams that transcode first often find that the transcode job becomes the bottleneck that delays the whole project by weeks. Processing in-place means you can begin indexing earlier content while later content is still being ingested.
Timecode: The Biggest Practical Problem
Timecode is how you link a search result to a specific location in a media file. When a user finds a clip at timestamp 01:14:32:12 in a search result, they expect to open that file in an NLE and land on exactly that frame. Timecode discontinuities break this contract.
In this archive, approximately 22% of files had at least one timecode discontinuity — a break in the continuous counter, typically from recording interruptions, tape restarts, or conversion artifacts from older digitization workflows. Some files had timecode sequences that reset multiple times within a single file.
There are two approaches to handling this. The first is to reject files with discontinuities and flag them for manual remediation. This is clean but means large portions of an archive become unsearchable until someone fixes the files — which may never happen. The second approach, which we use, is to build a timecode normalization pass that maps file-internal discontinuous timecode to a reconstructed continuous position reference, stores both the original timecode and the normalized position, and returns results using the original timecode for NLE compatibility while using the normalized position internally for search index alignment.
This works well in most cases. The edge case where it breaks down is files where the timecode is not only discontinuous but actively incorrect — where the recorded timecode doesn't correspond to the real-world recording time at all. In those cases, the normalized position is reliable but the returned timecode value will be wrong, producing a misleading NLE navigation offset. We surface these files explicitly in the index metadata so users know which results require manual timecode verification.
Parallelization and Queue Management
Processing 40,000 hours requires parallelization. The pipeline — audio extraction, ASR, diarization, OCR on keyframes, visual classification, embedding generation — has different computational characteristics at each stage. ASR is the most compute-intensive step per unit of audio; visual classification is the most I/O-intensive due to frame extraction; embedding generation has high memory requirements for batch processing.
We run these stages as separate queue-backed workers rather than as sequential per-file operations. A file enters a work queue, each stage processes it when capacity is available, and outputs are written to staging storage before being merged into the final index. This means the bottleneck stages can be scaled independently: during the initial 40,000-hour ingest, ASR workers were the constraint, and we ran eight concurrent ASR jobs while other stages were underutilized.
One thing we didn't anticipate: queue failures from files that caused worker crashes rather than recoverable errors. A small number of files — roughly 0.4% — had corruption at the codec level that caused decoder hangs rather than decode errors. A decoder hang doesn't raise an exception; it just stops the worker. Without explicit timeout handling per file, these files silently killed worker processes and left jobs stalled in the queue indefinitely. We now run per-file watchdog timers at every stage with a hard kill threshold and automatic dead-letter queue routing for files that exceed the threshold.
Index Quality Verification
Running a pipeline across 147,000 files produces a lot of output. The question is how to verify that the output is correct. Manual spot-checking 147,000 index entries isn't feasible; you need systematic quality metrics.
We track three categories of quality signals. First, coverage: what percentage of files have complete index entries across all modalities (ASR transcript, OCR results, visual embeddings)? A file with missing ASR coverage means its audio content is unsearchable. Coverage gaps usually indicate processing failures that weren't properly caught and routed to the dead-letter queue.
Second, ASR confidence distribution: for each processed file, we retain the average ASR confidence score across segments. Files with low average confidence are likely to have high WER and should be flagged as lower-reliability results. We surface confidence scores in search results so users can weight their expectations accordingly.
Third, timecode coherence: for each file, we verify that the returned hit timecodes are within the file's known duration and don't fall in known discontinuity zones. Results that fail this check are flagged but not suppressed — the underlying content may still be relevant, but the navigation offset may be imprecise.
The archive is not a finished product. We're not saying this is a solved problem — there are always files that fall through the quality checks, formats that behave unexpectedly, and content that resists automated indexing. What we can say is that after running this pipeline to completion on 40,000 hours, approximately 94% of files have searchable content in at least one modality, and 87% have full coverage across all three. That's not 100%, and we document the gaps. But it's a material improvement over the prior state, which was 0%.
