Broadcast AI inference placement: edge, on-prem or cloud?

A speech-recognition job does not need every pixel of the programme it is captioning. Yet poor broadcast AI inference placement can move full video across a network just to reach the GPU that processes its audio.

Decide what media the job needs, when its result must arrive and what should happen if it misses its deadline. Then choose the compute location.

The EBU published version 2.0 of its Dynamic Media Facility reference architecture on April 15, 2026. It describes software media functions that can be deployed locally, at remote locations or in public cloud environments. That expands the available choices; it does not make those locations operationally interchangeable. EBU reference architecture publication

The practical answer is usually workload-specific. A software-defined plant can have a local live-inference path, a shared GPU pool and cloud-based asynchronous work without forcing every job through the same place.

Documentation and capabilities checked August 30, 2026. Numerical examples below are illustrative engineering calculations, not measured Amira performance.


AI-generated conceptual illustration. This is not an actual facility, product interface or implementation diagram.

What does a container change about inference placement?

A container packages the application and its dependencies. The host still has to supply compatible hardware, networking and enough sustained capacity for the job.

The EBU's architecture separates infrastructure, host and container platforms from the media functions they support. It describes containers running on hosts within clusters, with resource planning and network support below the application layer. That separation is useful because the same application package may face very different operating conditions at different sites. EBU DMF reference architecture v2.0

Kubernetes, for example, documents a device-plugin route for exposing GPUs to containers. It requires suitable vendor drivers and the corresponding plugin on the nodes. A running container is therefore insufficient evidence that the intended accelerator is available. Kubernetes: schedule GPUs

Before calling a workload portable, test its media input, accelerator access, model loading, output timing and recovery behaviour on each target environment. Include any CPU-based decoding and format conversion. Those operations can become bottlenecks even when the GPU is lightly loaded.

Keep the question concrete: can this deployment sustain this media workload, with this deadline, during the failures the service must tolerate?

Which inference jobs belong at the edge, on-premises or in the cloud?

Start with the location that meets the workload's deadline and media-access requirements with the fewest fragile dependencies.

Here, edge means compute close to the source or operational endpoint, such as a venue or local production system. An on-premises cluster is a shared pool within a broadcaster-controlled facility. Cloud means provider-hosted compute. These categories can overlap: an edge system may be managed remotely, and a broadcaster may operate its own software in a cloud account.

The following is a decision framework, not a claim that any location is inherently broadcast-safe:

For a producer, placement affects when a usable transcript or clip description becomes available. For master control, it affects the alarm and fallback procedure. For a VP of engineering, it determines which team owns the service when something breaks.

Record those responsibilities alongside the topology. An unnamed owner is an architectural dependency too.

How much media actually needs to reach the model?

Send the smallest representation that preserves the information required by the task. Verify that reduction against quality requirements before relying on it.

SMPTE ST 2110 carries separately timed video, audio and ancillary-data streams. That separation allows different media components to take different routes; inference does not automatically require transporting the whole programme bundle. SMPTE: ST 2110 standards suite

Consider an illustrative uncompressed 1080p50 source with 10-bit 4:2:2 video and 48 kHz, 24-bit stereo audio. In 4:2:2 sampling, two pixels share two chroma samples alongside two luma samples, giving 40 bits per two pixels at 10 bits per sample. The active-video payload is therefore 20 bits per pixel. RFC 4175: uncompressed-video payload format

The arithmetic is:

• Active video: 1920 × 1080 × 50 × 20 = 2,073,600,000 bits/s, or 2.0736 Gb/s.
• Stereo audio: 48,000 × 24 × 2 = 2,304,000 bits/s, or 2.304 Mb/s.

The video-only payload rate is 900 times the stereo-audio-only rate in this example. This is a comparison of different media components, not a compression gain. Both figures exclude transport headers, redundancy and other programme data; they are not network-circuit sizing recommendations.


Original Amira Labs calculation and graphic. Decimal units; illustrative uncompressed payloads. Video-dependent tasks still require suitable video input.

For speech recognition, selecting the appropriate audio channels near the source can change the transport decision substantially. For visual review, a proxy may be appropriate, but resolution reduction and frame sampling can remove the evidence the task needs.

Choose the representation per task. Do not let the convenience of one transport endpoint determine what every model receives.

How should you build a live-inference latency budget?

Measure from the relevant media event to the usable downstream result, including collection, transfer, queueing and presentation.

Model execution is only one component. NVIDIA's Triton documentation distinguishes request time, scheduler queue time, input processing, model execution and output processing. Those server-side metrics are useful, but they do not include every stage of a broadcast workflow. NVIDIA Triton: metrics

Suppose a team assigns an illustrative 1,500 ms event-to-result budget. Its planning allowances might be:

That leaves 300 ms of unallocated margin. These are chosen allowances, not a benchmark or a captioning standard. If the application needs more context before producing a stable result, the budget changes.

Validate the whole path under load. Do not add independently measured 99th-percentile stage latencies and label the sum an end-to-end 99th percentile. Measure the distribution of complete event-to-result traces, plus the proportion that miss the deadline.

Also define the output you are timing. A first partial transcript, finalised text and a formatted caption delivered to an encoder are different milestones. An alert that arrives after its operational decision point is still late, even if inference itself was fast.

How do media connections differ from orchestration?

Orchestration starts and configures services; the media path carries the content those services process. Both must work for inference to be useful.

Within AMWA's Networked Media Open Specifications (NMOS), IS-04 supports discovery and registration of network resources, while IS-05 handles device connection management. Those are valuable control capabilities. They do not, by themselves, demonstrate that an inference workload has adequate GPU capacity or meets its deadline. AMWA IS-04, AMWA IS-05

Map the complete path from source selection through media reception and inference to the output consumer. Verify source identity and timestamps across conversions so the result remains associated with the correct programme moment.

For the related connectivity discussion, see the MXL broadcast guide. Amira consumes MXL as a Sense tap source. It is one source option, not a reason to prescribe a particular inference location.

An operator needs more than a service marked “running.” Useful checks answer whether the expected source is present, media is advancing, results are current and the downstream system is receiving them.

Can live and background AI jobs share the same GPU pool?

They can, provided the allocation and queue policies protect live deadlines under the actual mixed workload.

There is an important distinction between sharing access and reserving capacity. NVIDIA's current GPU Operator documentation says time-slicing interleaves workloads on an oversubscribed GPU without memory or fault isolation between the shared replicas. Requesting additional time-sliced GPU resources does not guarantee a proportional amount of compute. Multi-Instance GPU (MIG), on supported hardware, provides partitioning with memory and fault isolation, subject to the chosen profile and platform. NVIDIA: time-slicing and MIG comparison

For a live service, ask whether its allocation is a dedicated GPU, a hardware partition or shared access. Then test it while background work is active. An empty-cluster demonstration leaves the central question unanswered.

Set admission rules: which jobs may start, which must wait, and which can be interrupted? Reserve recovery capacity against measured workload limits. A spare slot on paper is useless if the remaining hardware cannot meet deadlines after a failure.

A single GPU-utilisation percentage is too coarse for this decision. Track queue age, missed deadlines, memory pressure, media drops and throughput over a representative programme schedule. The broadcast AI GPU buyer's guide covers hardware selection; placement adds contention and operational ownership to that calculation.

What happens when an inference node or connection fails?

The service needs a defined recovery path and a visible state for results that were delayed, lost or never produced.

Restarting a container does not recreate the live media that passed during the outage. Depending on the application, recovery may require buffered input, a new streaming session, rebuilt context and protection against duplicate results. Decide explicitly whether work is replayed, skipped or handed to another service.

Kubernetes distinguishes startup, liveness and readiness probes. Readiness can account for initialisation work such as loading files or warming caches. Your application must still define what “ready” means for receiving its media and producing usable results. Kubernetes: container probes

Redundancy also needs physical meaning. Kubernetes topology-spread constraints can distribute workloads across failure domains such as nodes and zones. Two copies sharing the same host, power dependency or ingress bottleneck may still fail together. Kubernetes: topology spread constraints

Before deployment, write the operator's actions for WAN loss, stale output and a failed inference instance. Include how the control room recognises recovery. Avoid automatic failback that can oscillate between unhealthy paths.

Test these procedures in an isolated environment or an approved shadow trial. Do not introduce a live-output interruption simply to discover whether the fallback works.

When is cloud the better placement, and where does this guide stop?

Cloud is a strong candidate when it avoids moving an already-cloud-hosted source, handles authorised burst work or supplies capacity for tasks whose results can wait.

Archive enrichment, programme summaries and experiments can have different deadlines from live captions or operational alerts. Conversely, a source-adjacent service may be a poor fit for an occasional large job if local hardware and support would sit idle between runs. Measure the actual duty cycle and include operations effort in the comparison.

Treat connectivity, encryption and permission to process as separate checks. AWS states that Direct Connect does not encrypt traffic in transit by default; customers must configure an appropriate encryption option. A private connection is therefore not sufficient evidence of an encrypted media path. AWS: Direct Connect encryption

This guide does not establish contractual permission to transfer content, jurisdiction-specific compliance or a guaranteed capacity for any model. Obtain those approvals separately. Local deployment also needs access controls and secure operations.

Finally, correct placement does not establish correct output. Recognition errors, visual misses and editorial decisions require their own evaluation and supervision. A result can arrive on time and still be wrong.

What should a broadcast team do this week?

Map one real workload end to end before committing to a plant-wide inference topology.

1. Write a one-page workload contract. Name the source, required media representation, output consumer, deadline, operating schedule and service owner. Record which locations are permitted.
2. Compare two plausible placements. Use the same authorised test material and quality settings. Measure the complete result path, transfer volume and sustained capacity, including competing work.
3. Exercise the failure policy. In a safe test, verify startup, loss of connectivity, node failure and recovery. Record what operators see and what happens to missing or duplicate results.
4. Approve a workload-specific placement. Keep the assumptions with the decision. Re-test when the model, input representation, workload mix or infrastructure changes.

Ask any supplier where the media is transformed, what happens when the queue reaches its limit, and whether the demonstrated spare capacity survives a real failure.

Those answers determine where inference belongs. The container image is only the starting point.

Sources

• EBU: Dynamic Media Facility reference architecture v2.0, published April 15, 2026
• EBU: full reference architecture v2.0 white paper
• SMPTE: ST 2110 standards suite
• RFC Editor: RFC 4175, RTP payload format for uncompressed video
• Kubernetes: scheduling GPUs
• AMWA: IS-04 NMOS discovery and registration
• AMWA: IS-05 NMOS device connection management
• NVIDIA: Triton inference-server metrics
• NVIDIA: GPU time-slicing and Multi-Instance GPU comparison
• Kubernetes: liveness, readiness and startup probes
• Kubernetes: pod topology spread constraints
• AWS: encryption in Direct Connect