[Typescript] Require high reliability structured output & streaming - possible?

API
1

2024-11-27T23:00:00Z

Reading the forum, the docs, the cookbooks and the API ref it look’s like I’m backed into a corner. I’m very happy with the performance of the Structure Output approach through openai.beta.chat.completions.parse(), but this method doesn’t currently support streaming!

Will streaming come to parse soon? Is there a limitation due to underlying parser that can’t approve the schema until it’s fully complete?

What are my options - going back to JSON and strongly typing the prompt, and just performing rigorous error testing and re-completions every time it fails? That isn’t sustainable.

const completion = await openai.beta.chat.completions.parse({
        model: "gpt-4o-2024-08-06",
        stream: false, // STREAM CAN ONLY BE FALSE
        messages: fullMessages,
        response_format: zodResponseFormat(ReportSchema, "report"),
    });

Example Schema trying to be forced: (The prod schema is a lot more complicated)

const ReportSchema = z.object({
    chatResponce: z.string(),
    reportMarkdown: z.string(),
    title: z.string(),
    description: z.string(),
});
2

I’m reimplementing this, hopefully the success rate is high enough:

const stream = await openai.chat.completions.create({
            model: "gpt-4o",
            messages: fullMessages,
            stream: true,
            response_format: { type: "json_schema", json_schema: schema }
        });