Streaming for AI Guardrails

The F5 AI Security platform sits between your application and upstream providers such as OpenAI, Anthropic, and Google. Before a response reaches your client, the platform runs F5 AI Guardrails scans on both the input and the output, per the security policy configuration.

With the instructions in this document, you can connect a streaming model to AI Security. AI Guardrails will work for these types of connections.

The buffering pattern

To use streaming, you need to understand relevant API requests. Specifically, you can use Scanners endpoints starting with Get guardrails.

Every API request follows this sequence:

1. Client sends a request. The full request body arrives at the platform.
2. Guardrails run on the input. All configured scanners evaluate the input text.
3. If cleared, the provider is called. The platform sends the request to the upstream AI provider.
4. The provider responds. The platform collects the full provider response. If the provider streams its response, the platform reads all chunks before proceeding.
5. Guardrails run on the response. If response scanning is configured, scanners evaluate the provider output.
6. A single JSON response is returned. The client receives the complete result.
7. AI Security sends back the provider response. (If the response is blocked, you'll see a relevant error message.)

The client sees no output until the process is complete.

Endpoints at a glance

The following endpoints relate to streaming in AI Security deployments:

Endpoint	Streaming response?	Client requests streaming?	What happens
OpenAI (POST /openai/{provider}/...)	No (buffered)	Yes (stream: true)	Collects the full streamed response, returns a single JSON payload.
Google (POST /google/{provider}/...)	No (buffered)	Yes (streamGenerateContent)	Collects the full streamed response, returns a single JSON payload.
Anthropic (POST /anthropic/{provider}/...)	No (buffered)	Yes (stream: true)	Collects the full streamed response, returns a single JSON payload.
Raw scan (POST /scans/raw/{format})	Yes (same chunks)	Yes (SSE content accepted)	Runs guardrails, returns the same chunks. Results in response headers.

Scan streaming

You don't have to route traffic through the provider-compatible endpoints to run guardrails. The /backend/v1/scans/raw/{format} endpoint lets you scan streaming request and response bodies directly. It accepts two format values: openai-responses-request for outgoing request bodies and openai-responses-sse for captured streamed responses. The endpoint doesn't call any upstream provider. It runs guardrails on the content you send, returns the same chunks it received, and reports results in response headers.

This endpoint currently supports OpenAI Responses API formats.

For step-by-step instructions, see Scan streaming.

What clients see

• Internal API consumers (scan and prompt endpoints) receive a single JSON object after all processing completes. There is no visibility into intermediate stages, for example whether scanning has finished and the provider call is in progress.
• Provider SDK users (OpenAI, Google, Anthropic endpoints) pass stream: true or equivalent in their requests, but the response they receive is still a single buffered JSON body, not the native SSE stream they would get from the provider directly. This breaks the real-time typing experience these SDKs are designed to provide.
• Raw scan endpoint users receive the same chunks back that they sent in the request. Scan results appear in response headers (x-ai-security-outcome, x-ai-security-scan-id). Because no upstream provider is called, the response contains only the content the client sent.
• UI users are the only group with real-time visibility, using the Prompts WebSocket. They see staged events as a prompt moves through the pipeline.