ModelBehaviorError: Invalid output type

API
,
1

Hello everyone. I use OpenAI Agents SDK but I have a problem. Even if I define an outputType, model rarely gives outputs don’t align with the ouputType defined. This causes my backend service to go down because of an unhandled exception.

export const Answer = catchAsync(async (req: CustomRequestChatbot, res: Response) => {
  
...
...
...
 const runner = new Runner({
      workflowName: 'mas_storefront',
      groupId: chatSession.sessionId,
      traceId: trace_${message.id},
      tracingDisabled: false
    })

    const result = await runner.run(
      startAgent,
      message.message.content,
      {
        context: context,
        previousResponseId: chatSession.lastResponseId,
        stream: true,
      },

    );
    
    for await (const event of result.toTextStream()) {
      res.write(event);
    }

    res.end();
...
...
...
}

Unhandled rejection ModelBehaviorError: Invalid output type
    at executeToolsAndSideEffects (/app/node_modules/@openai/agents-core/src/runImplementation.ts:614:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Runner.#runStreamLoop (/app/node_modules/@openai/agents-core/src/run.ts:781:30) {

catchAsync doesn’t catch the error. I also tried to wrap .run() and .toTextStream() blocks with try/catch before, but it also didin’t work. I couldn’t find a way to handle this error. My backend is crushing every time API returns a response in the wrong format.

Also is it normal for API to give out of format response occasionally?

2 Likes
2

Hi, what is the format it must return?
Is your error intermittent or stable on specific formats?
Is it the same model that must provide this format?

Thanks.

3 Likes
3

Actually the format is always the same and defined by zod object as OpenAI Agents TS SDK suggests. However, sometimes the response is a string which causes the error.

export const AgentsResponseStorefront = z.object({
    content: z.string(),
    suggestive_answers: z.array(z.string()),
    recommended_products: z.array(z.string()),
    forms: z.array(z.object({
        type: z.string(),
        fields: z.array(z.object({
            name: z.string(),
            type: z.string(),
        }))
    }))
})
1 Like
4

See GPT4.1 doesn't follow strict json schema

1 Like
5

Do you guys use field description in the schemas? I do because it helps focus the model on the exact values.

Also for the schema in the example looks like open door for hallucinations if not detailed in the system prompt (too loose as definition).

But having no other info makes me guess more than help.

1 Like
6

I don’t think it is possible to give descriptions in zod schema. Their descriptions are in the system prompt. However, this is not important because it should give an output following the schema type. Descriptions may only effect the content’s relevance.

1 Like