Streaming Structured Output using Assistants API

API
, ,
1

Hi everyone,

I’m using Structured Output in streaming with the Assistants API and noticed that the Chat Completions API provides output deltas like this:

const streamobj = openai.beta.chat.completions.stream({
    model: "",
    messages: [
        { role: "system", content: "" },
        { role: "user", content: text }
    ],
    stream: true,
    response_format: zodResponseFormat(MySchema, "myschema")
})        
.on("content.delta", ({ snapshot, parsed }) => {
    console.log("Raw Content:", snapshot);
    console.log("Parsed Content:", parsed);
});

These deltas allow me to access both:

  • Raw content: The JSON stream as-is.
  • Parsed content: The progressively completed JSON up to the latest chunk.

To handle streaming JSON from the Assistants API, I’ve implemented a function that “parses”/completes the JSON as it arrives.

However, I couldn’t find any mention of similar deltas in the Assistants API. The documentation on Structured Output for Assistants is almost nonexistent.

Does anyone know if there’s an official solution for handling streaming in structured output in the Assistants API? Or should I just continue using my own function?

Thanks!

2

If you parse the JSON output on your own, you can use the on('messageDelta', (delta: MessageDelta, snapshot: Message) => ...) method. More information can be found in the helpers.md file in openai/openai-node github repo.