Hey everyone. I am trying to use the chat completions API with streaming and tool calls. I can see the tool arguments in each delta but I cannot see the tool call id nor the tool name. In any of the chunks. I’m unsure if this is a bug or I’m missing something.
As far as I know, I need to have the tool call id so I can respond and continue sending messages correct?
Currently using the openai npm package v4.68.0 if it matters.
Could you provide some code snippets and possibly errors - if encountering any?
This will make it a lot easier to understand and help with your problem. 
I can share some of the code.
import OpenAi from "openai";
const client = new OpenAi({
apiKey: Config.openAiApiKey,
});
const oaiStream = await client.chat.completions.create({
model: "gpt-4o-mini",
messages,
tools,
tool_choice: "auto",
stream: true,
stream_options: {
include_usage: true
}
});
for await (const chunk of oaiStream) {
// Both toolId and functionName equals undefined for every chunk
const toolId = chunk.choices[0].delta.tool_calls[0].id
const functionName = chunk.choices[0].delta.tool_calls[0].function?.name
}
Besides configuration, I don’t think my code is wrong. It generally works and I can have a conversation with the AI.
The issue is that the chunk.delta.tool_calls items do not return an id or function name. I only receive the arguments. I have seen people state that the id comes in on the first chunk, but I have stepped through every single chunk and there is nothing.
This is blocking me because without the function name, I don’t know which function to use locally, and without the tool id, I can’t continue the conversation with the AI.
Without streaming, the AI responses contain all the data needed for me to determine which function to invoke and the tool id to return to the AI. It just doesn’t give me the data while in streaming.
It is in the first chunk. Or chunk [0] if you will.
data = {
"id": "chatcmpl-xyz",
"object": "chat.completion.chunk",
"created": 1730391658,
"model": "gpt-4o-mini-2024-07-18",
"system_fingerprint": "fp_0ba0d124f1",
"choices": [
{
"index": 0,
"delta": {
"role": "assistant",
"content": None,
"tool_calls": [
{
"index": 0,
"id": "call_xZtWnTjjc0jhgtsMMpq6kC0w",
"type": "function",
"function": {
"name": "get_current_weather",
"arguments": ""
}
}
],
"refusal": None
},
"logprobs": None,
"finish_reason": None
}
],
"usage": None
}
# Extracting the chunk function name
function_name = data["choices"][0]["delta"]["tool_calls"][0]["function"]["name"]
print(function_name)
You might see if you were clever enough in your code before to discard the first chunk because it doesn’t have “content”.
Chunk function call dump for your understanding
data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","created":1730391658,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0ba0d124f1","choices":[{"index":0,"delta":{"role":"assistant","content":null,"tool_calls":[{"index":0,"id":"call_xZtWnTjjc0jhgtsMMpq6kC0w","type":"function","function":{"name":"get_current_weather","arguments":""}}],"refusal":null},"logprobs":null,"finish_reason":null}],"usage":null}
data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","created":1730391658,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0ba0d124f1","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\""}}]},"logprobs":null,"finish_reason":null}],"usage":null}
data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","created":1730391658,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0ba0d124f1","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"location"}}]},"logprobs":null,"finish_reason":null}],"usage":null}
data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","created":1730391658,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0ba0d124f1","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\":\""}}]},"logprobs":null,"finish_reason":null}],"usage":null}
data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","created":1730391658,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0ba0d124f1","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"Miami"}}]},"logprobs":null,"finish_reason":null}],"usage":null}
data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","created":1730391658,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0ba0d124f1","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\",\""}}]},"logprobs":null,"finish_reason":null}],"usage":null}
data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","created":1730391658,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0ba0d124f1","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"unit"}}]},"logprobs":null,"finish_reason":null}],"usage":null}
data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","created":1730391658,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0ba0d124f1","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\":\""}}]},"logprobs":null,"finish_reason":null}],"usage":null}
data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","created":1730391658,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0ba0d124f1","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"fahren"}}]},"logprobs":null,"finish_reason":null}],"usage":null}
data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","created":1730391658,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0ba0d124f1","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"heit"}}]},"logprobs":null,"finish_reason":null}],"usage":null}
data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","created":1730391658,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0ba0d124f1","choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"\"}"}}]},"logprobs":null,"finish_reason":null}],"usage":null}
data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","created":1730391658,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0ba0d124f1","choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"tool_calls"}],"usage":null}
data: {"id":"chatcmpl-xyz","object":"chat.completion.chunk","created":1730391658,"model":"gpt-4o-mini-2024-07-18","system_fingerprint":"fp_0ba0d124f1","choices":[],"usage":{"prompt_tokens":143,"completion_tokens":20,"total_tokens":163,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens_details":{"reasoning_tokens":0}}}
I have a class object that collects from the stream, whether repeated or appended, simply or needing more unwrapping. Simple version.
class ResponseState:
"""Holds the state of the streaming response."""
def __init__(self) -> None:
self.content: str = ''
self.function_call: Dict[str, str] = {}
self.tool_calls: Dict[int, Dict[str, Any]] = {}
self.finish_reason: Optional[str] = None
self.usage: Optional[Dict[str, Any]] = None
When [DONE], the “content” has already been printed, and action can be taken on either functions or tools.
So I just debugged again and it was there. I am literally not kidding when I said I have restarted and stepped through all my chunks multiple times only to see nothing. But now its there.
I don’t know why its there now after reading your post but its there and I’m not gonna complain… I think I need sleep. Or coffee. Or both.
Thanks all!