500-feed monitoring economics: what does incident evidence cost?

Model evidence retention across 500 broadcast feeds. Compare continuous recording, event clips and local buffers, and budget for retrieval and incident peaks.

Kyle Suess8 min read

Share

Share on LinkedIn
Three side-by-side cards compare continuous recording, selected incident clips and local circular buffers for a 500-feed estate.

Five hundred live feeds can fit comfortably on a monitoring dashboard. Press record on all of them and the same estate can generate 324 terabytes of proxy video in a 30-day month. That is before a second copy, before anyone downloads a clip, and before an engineer finds out whether the recording contains the audio track that went missing.

The economics of 500-feed monitoring change once the budget includes incident evidence. A probe answers whether it saw a fault. An evidence system has to show what happened, where it happened, and which viewers could have experienced it. A few minutes of video can settle a dispute quickly. A few minutes from the wrong side of the encoder can send an investigation in the wrong direction.

What evidence does a 500-feed operation need to keep?

Start with the decision each record supports. An operator investigating missing captions needs the caption service, the corresponding audio, and enough lead-in to tell when the failure began. An engineer isolating packet loss may need transport counters and a packet capture from specific boundaries. A distributor resolving a rights complaint might need the actual output from one territory and a trustworthy timestamp. A single always-on preview rarely answers all three questions.

Write an evidence policy for each combination of service and failure class. Identify the observation point, time span, required tracks, and retrieval deadline. Then decide who is permitted to see the record and who can extend retention after an incident. If only the output from a customer-facing endpoint establishes impact, retaining an upstream preview has limited value for that purpose. It can still help find the cause when captured alongside the endpoint evidence.

Three retention mechanisms cover different needs. A continuous archive preserves everything in a defined window. An incident clip preserves a selected interval once a trigger fires. A circular buffer holds a short moving window and overwrites its oldest media unless an operator or a rule promotes part of it. Many facilities will need more than one mechanism. Their budgets should show the scope of each explicitly, so a continuity requirement on a handful of critical services does not automatically become a fleet-wide recording mandate.

How many bytes does each retention policy create?

At a constant 2 megabits per second, an illustrative proxy consumes 0.9 decimal gigabytes per feed-hour. Multiply bitrate in megabits per second by 3,600 seconds, divide by eight to convert bits to bytes, then divide by 1,000 for decimal gigabytes. At 500 feeds, the rolling write rate is 450 GB an hour. The examples below assume one copy at that bitrate, with no protocol overhead or metadata. Real audio, captions, additional views, and variable bitrates can change the result.

Policy

Modeled workload

Media volume

Continuous 30-day archive

500 feeds × 720 hours × 0.9 GB

324,000 GB in steady-state retention

Continuous seven-day archive

500 feeds × 168 hours × 0.9 GB

75,600 GB in steady-state retention

Six-hour circular buffer

500 feeds × 6 hours × 0.9 GB

2,700 GB of active buffer capacity

Thirty days of event clips

500 feeds × 3 clips/day × 30 days × 0.06 GB

2,700 GB of retained clips

The event example assumes each clip lasts four minutes and there is no deduplication or overlap. It produces 45,000 objects a month if there is one object per clip. Two separately uploaded observation points double both numbers. The six-hour buffer happens to hold the same volume as the 30-day clip library. It still writes 324,000 GB over the month because it continuously replaces old media. That distinction matters when sizing local disks, placing buffers, and pricing remote writes.

Use a clearly stated rate for comparisons. At an assumed $0.025 per decimal GB-month, storing one steady-state 30-day continuous copy would be $8,100 per month. A seven-day archive would be $1,890, and the 30-day clip library would be $67.50. Those are storage-only worksheet values at a hypothetical flat rate. A real quote may use binary units, price tiers, minimum billed object sizes, and separate request or retrieval fees. If the buffer lives on local disks, use the cost of installed and usable capacity, measured write throughput, and replacement instead of applying the object-storage rate.

When does a small archive generate a large bill?

A small archive can still be expensive when objects move or get read frequently. Price the path from capture to the incident reviewer. Each step can have a different charging unit: objects written, retained bytes, data retrieved from a storage class, bytes crossing a network boundary, and compute to turn evidence into playable video. A policy that creates many small records may cost more in requests than a simple GB comparison suggests.

Amazon S3’s class documentation, checked November 6, illustrates the trade-off. S3 Standard-IA has a 30-day minimum storage duration and a 128 KB minimum billable object size, with retrieval charged by volume. AWS also documents that the default lifecycle behavior blocks transitions for objects under 128 KB unless the configuration overrides it. A 60 MB clip and a tiny event manifest therefore need separate rows. Before moving either to a colder tier, test the actual lifecycle configuration, including what happens to prior object versions and copies kept under a hold.

Include an expiry test in the same exercise. Take a known clip past its intended retention window, then attempt to retrieve it with ordinary and privileged credentials. Verify the behavior of replicas, exported copies, and versions. A policy can appear correct at the front door while older copies continue to consume capacity and remain available elsewhere.

Calculate steady-state retrieval and a bad week separately. A compliance review might pull hundreds of hours for one program. An external support vendor might access the same files from another region. A player that reads a few seconds can still trigger multiple object requests. Ask a reviewer to retrieve representative footage using the exact interface, identity, and network they will use during an incident. Measure elapsed time and billable usage together.

Include labor explicitly. If each of the modeled 45,000 monthly clips required two minutes of review, that would consume 1,500 operator hours. At the same review time, inspecting 2% would take 30 hours. Both are illustrative. The workload should be measured after the alert correlation rules you plan to use. The master-control alarm correlation guide explains how to group dependent alarms while preserving evidence of distinct incidents. Group alerts for review, and preserve the underlying observation records so investigators can still see who was affected.

How large a burst can the system survive?

Size the evidence path for simultaneous faults. If every feed in the 500-feed example promotes one four-minute clip, that is 30 GB. At 100 Mb/s of dedicated usable throughput, the transfer takes about 40 minutes before overhead or retries. At 1 Gb/s it takes about four minutes. An upload queue that routinely competes with contribution or distribution traffic may take longer.

The risk is highest when a shared fault takes out the evidence path as well as the service. Reproduce a loss of connectivity while clips are being promoted. Confirm that buffers keep writing, that the job queue resumes from the correct interval, and that retried uploads do not create multiple retained copies. Use an identifier tied to observation point and time interval. A receiver should be able to tell whether a clip was overwritten, still waiting, uploaded successfully, or retained in a different location.

Retention starts before a detection rule fires if you need the lead-up. A buffer window has to cover the required lead-in, the delay before detection, any human escalation delay, and the time until promotion can safely read the original media. Protect high-priority intervals from overwrite while they wait for export, and bound the amount of disk a single outage can pin. Test what happens when those two policies collide. An intentional priority rule is preferable to discovering that the filesystem chose which evidence survived.

When is continuous recording the right purchase?

Continuous recording makes sense when staff cannot know in advance which time range they will need, or when a contract or compliance policy requires a complete record. It also helps when the alert rule misses the very fault a viewer reports later. Price the entire retention window, access controls, export capabilities, and the specific media quality required for the record. A low-rate proxy can be adequate for timing a black screen while losing detail needed to inspect a single corrupted frame or validate audio channel mapping.

Event clips earn their keep where the detection boundary is trustworthy and a short interval answers the question. Check for incidents that arrive through a different route, such as a complaint with no corresponding alarm. If operators need to go back further than the buffer, decide which services deserve a longer window. A smaller always-on archive for the highest-impact services can coexist with shorter buffers and event clips elsewhere.

The cheapest store is a poor fit if the recovery deadline is shorter than the access time or if a regional copy fails with the original. It is equally easy to overpay for full-resolution duplication when timestamps and a short selected proxy would settle the incident. The acceptance criterion is whether the right person can retrieve a record that answers the specific question within the agreed time.

What should the monitoring team measure this week?

Pick one quiet day and one event-heavy day. Export clip counts, retained bytes, capture points, reviewer minutes, and latency from fault to playable evidence. Keep continuous writes, committed storage, and exported bytes in different columns. Tag a period of missing telemetry explicitly so a quiet dashboard cannot masquerade as a quiet operation.

Then simulate a shared fault and request the same incident from two points in the chain. Have the reviewer find it without help from the capture engineer. Compare the clip’s clock, labels, tracks, and exact interval to the original incident timeline. Time the restore from the coldest class you plan to use. Ask whether the file would settle the decision it was retained to support.

Take the measured rates to procurement with your retention assumptions next to them. Ask suppliers to quote the normal month and the burst month, to identify who owns the original clips and exports, and to name the unit charged for every copy. That is how a single line called monitoring storage becomes a budget someone can defend.

Sources