Responses & streams

Inference on sference is modeled as a response resource: you create it, observe status, and read structured output (text, optional reasoning, errors). A response can be realtime (blocking, returns when inference finishes) or async (background: true, polled or streamed to completion). When you need incremental delivery, pair responses with streams and the events API.

Mental model

1. POST /v1/responses: enqueue work (model, input[], optional sampling knobs, metadata).
2. GET /v1/responses/{id}: poll status, output, and error until the job reaches a terminal state.
3. GET /v1/responses/events: cursor through SSE-style event history when you want a feed instead of polling the full document every time.
4. Streams (via the Python SDK / CLI): allocate a stream id to group related responses and fan out chunks independently of the JSON document.

If you are coming from the OpenAI Responses API, the verbs should feel familiar; sference adds async windows and European residency guarantees on the infrastructure side.

When to use streams

Use streams when:

• UX demands partial tokens or progressive rendering.
• Downstream systems already speak chunked HTTP or SSE.
• You want to decouple transport from the final aggregated response record.

For massive static backfills, prefer Batch inference; it is cheaper, simpler to retry, and easier to reconcile in compliance exports.

Processing modes

• realtime: sync /v1/chat/completions, /v1/messages, or blocking POST /v1/responses (no window field). Deadline = now; the connection waits for the result.
• flex: /v1/chat/completions or POST /v1/responses with service_tier: "flex": discounted tokens in exchange for lower scheduling priority. (/v1/messages has no service_tier field and always runs realtime.) Flex requests may queue significantly longer than realtime (up to 30 minutes), so raise client timeouts; a flex request that cannot complete in time fails with 408 (OpenAI SDKs retry automatically). Flex is enabled per account; contact us to turn it on.
• async (24h): background responses (background: true, optionally metadata.completion_window: "24h", the only window value), SDK/CLI streams, and batches (window on create). Queued work scheduled to complete within the window.

All three modes share the same models, endpoints, and request shapes; only scheduling priority and price differ. Responses echo the tier actually served in service_tier.

Next steps