From Real-Time Streaming (RTSP) to Agentic Protocols

What Today’s Agentic Systems Can Learn from 90s Streaming Media

In 1996, the commercial internet was structurally ill-equipped for real-time media.

Bandwidth was constrained. Packet loss was common. Connections were unreliable. And the World Wide Web largely operated through stateless request-response interactions over HTTP.

At RealNetworks, I worked on the architecture behind RealMedia and participated in the IETF working group that standardized the Real-Time Streaming Protocol (RTSP, RFC 2326). We were trying to solve what was, at the time, a new class of internet-scale systems problem: how do you establish and maintain dynamic session state across unreliable networks, separate orchestration from high-volume data movement, and adapt continuously to changing network conditions?

Nearly thirty years later, as I work with modern AI systems and watch agentic architectures evolve, many of those challenges feel remarkably familiar.

The technologies are obviously different. An LLM reasoning over a tool call is not a media player receiving packets over RTP. But many of the underlying systems problems are strikingly similar: latency, state synchronization, expensive data movement, resource allocation, partial failure, interoperability, and recovery.

And I see AI architectures beginning to repeat some of the mistakes distributed systems learned to avoid decades ago.

Many agentic systems are still designed as sequences of API calls or increasingly elaborate prompt loops. That works surprisingly well for prototypes. It becomes much less attractive when the system must operate reliably, economically, and at scale.

The architectural patterns that helped make real-time internet streaming commercially viable offer several useful lessons for building enterprise AI platforms today.

1. Decouple the Control Plane from the Data Plane

One of the important architectural ideas behind RTSP was the separation of control from media transport.

RTSP handled session-level operations such as SETUP, PLAY, PAUSE, RECORD, and TEARDOWN. The high-volume media itself could travel separately, typically using RTP. The control plane negotiated capabilities and maintained session state; the data plane was optimized for moving media efficiently.

That separation seems obvious in retrospect. At the time, it was fundamental to making streaming practical.

I see the same architectural boundary becoming important again in AI systems.

Many current agent pipelines pass everything through the LLM context: orchestration instructions, routing decisions, tool schemas, conversation history, retrieved documents, database results, and sometimes very large raw payloads.

The context window effectively becomes both the control plane and the data plane.

That is convenient. It is rarely efficient.

Passing large documents, tabular results, media artifacts, or intermediate datasets repeatedly through reasoning agents increases latency, consumes context, raises inference cost, and creates unnecessary opportunities for information loss or transformation.

The better architecture is familiar:

Separate orchestration from data movement.

Agents should exchange lightweight, structured messages describing intent, capabilities, references, permissions, and execution state.

Heavy artifacts should remain outside the reasoning path---in object stores, databases, shared memory, vector stores, or other purpose-built systems---and be referenced through durable identifiers or resource handles rather than repeatedly serialized through prompts.

The LLM should reason about the data.

It does not necessarily need to carry the data.

2. State Must Be Explicit, Negotiated, and Resumable

Streaming exposed another weakness in early internet architecture: stateless interactions were poorly suited to long-lived experiences.

If every interruption meant starting playback from the beginning, streaming would never have been usable.

RTSP therefore introduced explicit session semantics. Client and server maintained shared state around the media session and its timeline.

That same distinction matters enormously in agentic AI.

Today, state in many AI applications is effectively an accumulated conversation string.

That works until the workflow becomes operational.

Imagine an eight-step agent workflow. The system retrieves data, performs analysis, calls several tools, generates an artifact, validates it, and finally updates an external system.

At Step 7, an API times out.

What happens?

In too many implementations, the workflow either fails entirely or starts again from the beginning---repeating expensive reasoning, potentially repeating side effects, and wasting both time and compute.

Production agent systems need something stronger:

Explicit, checkpointed state machines.

Each meaningful execution stage should produce durable state. Operations that can create side effects should be designed for idempotency. And the orchestration layer should be capable of resuming a session from a known checkpoint without replaying every upstream reasoning step.

The prompt history can inform the workflow.

It should not be the workflow.

This distinction is beginning to appear in emerging agent protocols as well. The 2026 revision of the Model Context Protocol (MCP), for example, moved away from requiring a persistent protocol session. Each request can stand on its own, while applications that need continuity can carry explicit state handles across calls.

That is an important architectural distinction.

Stateless protocols do not require stateless applications. They require state to live at the right layer.

For long-running agent workflows, the same principle should extend beyond individual tool calls: execution state, checkpoints, side effects, and recovery points should be explicit enough that work can resume without reconstructing the workflow from conversation history.

3. Adapt Compute to the Workload

Early internet streaming had to operate across wildly unpredictable network conditions.

Throughput could change dramatically within seconds. A system that assumed constant bandwidth would simply stall.

The solution was adaptation.

Streaming engines learned to vary quality and bitrate based on available network capacity. When conditions deteriorated, the system degraded quality rather than abandoning playback altogether.

AI infrastructure needs an analogous capability.

Many current systems effectively operate at one computational bitrate: every problem is sent to an expensive frontier model regardless of whether the task requires sophisticated reasoning or simple deterministic processing.

A regex extraction does not require the same computational machinery as strategic synthesis. Neither does schema validation, routing, classification, or many routine tool operations.

AI platforms should therefore treat inference as a dynamically allocated resource.

Match the computational path to the complexity of the task.

Deterministic code should handle deterministic problems.

Compact or specialized models can handle classification, extraction, routing, and other constrained tasks.

Large reasoning models should be reserved for situations where ambiguity, synthesis, judgment, or complex planning actually justifies their latency and cost.

The analogy to adaptive streaming isn't that tokens are packets or models are codecs.

The deeper principle is that infrastructure should continuously adapt resource consumption to workload requirements rather than provision every interaction for the worst case.

4. Open Protocols Outlast Proprietary Silos

The early streaming ecosystem was fragmented.

Vendors developed competing players, servers, formats, protocols, and proprietary extensions. That is normal in the early stages of a technology cycle: innovation moves faster than standardization.

Eventually, however, infrastructure benefits enormously from shared interfaces.

RTSP was standardized through the IETF, and parts of the broader streaming ecosystem became increasingly open and interoperable. RealNetworks itself later open-sourced major portions of its multimedia platform through Helix.

I see AI infrastructure moving through a similar phase today.

Agent frameworks, tool interfaces, memory systems, orchestration layers, and model APIs are proliferating rapidly. Most are optimized around their own abstractions.

That fragmentation is useful while the industry is experimenting.

We are already seeing movement toward common interfaces through protocols such as the Model Context Protocol (MCP). What began primarily as a standard way for AI applications to connect with external tools and context is expanding to cover more of what production agent systems need: capability discovery, authorization, long-running tasks, and extensibility.

Its evolution is itself instructive. MCP's 2026 revision moved its protocol core toward statelessness---requests no longer depend on a persistent session with a specific server, so any instance can handle any request without shared session infrastructure. It also formalized how tools describe their own capabilities and how long-running work can continue without holding a connection open. These are familiar distributed-systems concerns appearing again as an emerging protocol encounters production-scale requirements.

The specific protocols will continue to evolve. The direction matters more: as agentic systems become infrastructure, shared interfaces reduce proprietary integration and let tools and applications evolve independently.

Protocols evolve. Architectural constraints are more persistent.

Interoperability becomes particularly important when enterprises begin operating hundreds or thousands of agents, tools, models, and workflows across organizational boundaries.

At that point, interoperability becomes infrastructure.

I expect open standards around tool discovery, capability negotiation, identity, authorization, context exchange, and cross-agent handoffs to become increasingly important as agentic systems mature.

Enterprises will want to change models without rewriting applications.

They will want agents built by different teams to cooperate.

They will want tools to advertise capabilities through well-defined interfaces.

And they will want security and identity boundaries that do not depend on whichever agent framework happened to be fashionable when an application was built.

The exact standards are still evolving.

The architectural requirement is not.

5. Design for Degradation, Not Just Success

There is another lesson from streaming that may be even more important for AI.

We never had the luxury of designing for a perfect network.

Packets disappeared. Connections dropped. Bandwidth collapsed. Servers became unavailable. Clients behaved unpredictably.

Failure wasn't an exceptional state.

It was part of the operating environment.

So successful streaming architectures had to ask a different question:

When something fails, what is the best experience the system can still deliver?

AI platforms need the same mindset.

A production agent may depend on an LLM, retrieval service, vector store, authentication system, multiple APIs, external tools, and other agents. Any one of them can be slow, unavailable, rate-limited, or simply return something unexpected.

And unlike traditional software components, the reasoning layer itself is probabilistic.

That means reliability cannot be bolted onto an AI application after the prototype works.

It has to be part of the architecture.

Timeout budgets. Bounded retries. Circuit breakers. Fallback models. Tool substitution. Partial-result handling. Checkpointing. Idempotency. Observability. Graceful degradation.

The objective shouldn't be to pretend failures will not happen.

It should be to contain them.

A sophisticated AI system might fall back from a frontier model to a smaller model. It might skip an unavailable enrichment service. It might return a useful partial result and complete the remaining work later. Or it might pause an execution and resume it safely when a dependency recovers.

The exact mechanism depends on the application.

The principle does not:

Reliability is not the absence of failure. It is the ability of the system to behave predictably when failure occurs.

The Future Keeps Rediscovering the Past

There is a temptation whenever a new computing paradigm emerges to assume that its architectural problems are entirely new.

Some are.

Many aren't.

The systems we built for real-time media in the 1990s operated under very different constraints from today's AI platforms. But they forced us to confront problems that distributed systems engineers continue to encounter: state, latency, expensive data movement, resource constraints, partial failure, interoperability, and recovery.

AI introduces an extraordinary new variable into that equation:

probabilistic reasoning.

That makes the engineering problem more interesting, not less dependent on good systems architecture.

As agentic AI moves from prototypes into enterprise infrastructure, I believe the winning architectures will increasingly rediscover principles that have survived multiple generations of computing:

Separate control from data.

Make state explicit and recoverable.

Allocate expensive resources dynamically.

Build around interoperable interfaces.

Assume components will fail---and design the system to keep functioning when they do.

Thirty years ago, the challenge was making continuous media work over an internet that wasn't designed for it.

Today, we're trying to make intelligent, autonomous workflows operate reliably over infrastructure that wasn't designed for them either.

The protocols will be different.

The engineering discipline shouldn't be.