Function calling is not supported to Streaming Assistant

API
,
1

Now I am struggling how to implement function calling in streaming assistant run in my NestJS project. As you know, The example code is like this.
[https://platform.openai.com/docs/assistants/overview?context=with-streaming]

const run = openai.beta.threads.runs.createAndStream(thread.id, {
    assistant_id: assistant.id
  })
    .on('textCreated', (text) => process.stdout.write('\nassistant > '))
    .on('textDelta', (textDelta, snapshot) => process.stdout.write(textDelta.value))
    .on('toolCallCreated', (toolCall) => process.stdout.write(`\nassistant > ${toolCall.type}\n\n`))
    .on('toolCallDelta', (toolCallDelta, snapshot) => {
      if (toolCallDelta.type === 'code_interpreter') {
        if (toolCallDelta.code_interpreter.input) {
          process.stdout.write(toolCallDelta.code_interpreter.input);
        }
        if (toolCallDelta.code_interpreter.outputs) {
          process.stdout.write("\noutput >\n");
          toolCallDelta.code_interpreter.outputs.forEach(output => {
            if (output.type === "logs") {
              process.stdout.write(`\n${output.logs}\n`);
            }
          });
        }
      }
    });

It was able to get streaming working for regular text responses, but now I’m stuck on function calling. I would like to know about event handler of nodejs SDK for OpenAI. Is there anyone to know about that? especially function calling of streaming assistant?

2 Likes
2

Just to add to your question with my own, I’m in the same boat with working with nodejs. I also have streaming working, but I’m stuck on how to submit the tool outputs now that the logic has changed a bit with streaming with openai.beta.threads.runs.createAndStream(). In other words, the function below is how I was processing the functions and submitting them with non-streaming, but with the events that come with streaming, it isn’t 1:1, so I am a bit lost without clearer examples.

async function retrieveRun(threadId, runId) {
    let run;
    do {
        run = await openai.beta.threads.runs.retrieve(threadId, runId);

        if (run.status === 'requires_action') {
            const requiredAction = run.required_action;

            if (requiredAction.type === 'submit_tool_outputs') {
                const toolCalls = requiredAction.submit_tool_outputs.tool_calls;
                const toolOutputs = [];

                for (const toolCall of toolCalls) {
                    const functionName = toolCall.function.name;

                    const functionArgs = JSON.parse(toolCall.function.arguments);

                    const availableFunctions = {
                        get_weather: getWeatherData,
                        get_business_info: fetchBusinessInfo,
                    };
                    console.log("Function Name:", functionName);

                    const functionToCall = availableFunctions[functionName];

                    const functionResponse = await functionToCall(functionArgs);

                    const outputString = JSON.stringify(functionResponse);

                    toolOutputs.push({
                        tool_call_id: toolCall.id,
                        output: outputString,
                    });
                }
                console.log("toolOutputs: ", toolOutputs);
                await openai.beta.threads.runs.submitToolOutputs(
                    threadId,
                    runId,
                    { tool_outputs: toolOutputs }
                );
            }
        }
        await new Promise(resolve => setTimeout(resolve, 500));
    } while (run.status !== 'completed');

    return {
        status: run.status,
        tools: run.tools,
        file_ids: run.file_ids,
        usage: run.usage
    };
}
3

Here you go:
community openai com
/t/automatically-calling-tools-functions-with-assistants/758721/2

There is no runTools equivalent for assistant (at least I haven’t found it). :point_up: here is a solution you can use in your js code.