Streaming structured outputs field by field?

API
, ,
1

This is the relevant documentation: https://platform.openai.com/docs/guides/structured-outputs#streaming

You can use streaming to process model responses or function call arguments as they are being generated, and parse them as structured data.

That way, you don’t have to wait for the entire response to complete before handling it. This is particularly useful if you would like to display JSON fields one by one, or handle function call arguments as soon as they are available.

We recommend relying on the SDKs to handle streaming with Structured Outputs.

Using the Agents SDK you can set output_type of an Agent. So when you stream by token and print the deltas, you’ll see something like:

{“
field_1
“:”
value
“,”
field_2
“:”
value
…etc.

Can the SDK help stream token by token per field? Or you have to handle the parsing logic on the receiving end yourself?

2

The OpenAI SDK itself does not provide per-field streaming. If you are using Python, LangDiff (globalaiplatform/langdiff at GitHub) will be helpful. It parses a streaming structured output and then triggers per-field change handlers token by token.

3

From a developer perspective it doesn’t really matter, but the “ackshually” answer is that the output is by unified UTF-8 code points.

Take this emoji:

👨🏾‍👩🏻‍👧🏼‍👦🏽

Byte count: 41
Token count: 17
Tokens: 28823, 101, 52622, 122, 2524, 28823, 102, 100798, 2524, 28823, 100, 183024, 2524, 28823, 99, 52622, 121
Delta chunks: 1

That informs that even if you are buffering any ol’ arbitrary stream, you’d need a clever parser meant to deliver just key/value, which for nested JSON, likely means delivering complete root objects.

(this forum btw fails by translating that emoji into multiple shortcodes.)