same as the title.
if AsyncOpenAI + streaming request provided, api does not generate arguments. it only generate function name.
our service is stoped because of this.
hope this case fix ASAP
Just like “content” comes in token-sized packages,tool_call also need to be collected throughout a stream and appended, waiting until the final “finish_reason” is received (or then usage).
One way to do this is with streaming helpers, also including support of BaseModel as a function definition and validator.
import asyncio
import openai
# support response format as basemodel class
from pydantic import BaseModel
from enum import Enum
from typing import List, Union
class GetWeather(BaseModel):
city_name: str
us_state_abbr: str
client = openai.AsyncOpenAI()
def print_function_call(name: str, arguments: Union[dict, BaseModel]) -> None:
"""
Print a formatted string showing how a function (tool) was called.
If `arguments` is a Pydantic model, convert it using model_dump().
"""
if hasattr(arguments, "model_dump"):
arguments = arguments.model_dump()
arg_str = ", ".join(f"{key}={repr(value)}" for key, value in arguments.items())
print(f"\nFunction call: {name}({arg_str})")
async def main():
tool_call_name = None
final_tool_call_args = None
async with client.beta.chat.completions.stream(
model='gpt-4o-2024-08-06',
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Miami weather today?"},
],
tools=[
openai.pydantic_function_tool(GetWeather),
],
) as stream:
async for event in stream:
if event.type == 'content.delta':
# Use the 'delta' attribute to print the latest content chunk.
print(event.delta, flush=True, end='')
elif event.type == 'tool_calls.function.arguments.delta':
# If you wish to process partial function call arguments, do so here.
pass
elif event.type == 'tool_calls.function.arguments.done':
tool_call_name = event.name
final_tool_call_args = event.parsed_arguments # This might be an instance of GetWeather.
print_function_call(tool_call_name, final_tool_call_args)
if __name__ == '__main__':
asyncio.run(main())
Thanks for reply @_j
This one is not my code. it’s from official documents.
(my service is quite different from this one and it’s already in production status from few months ago)
throughout a stream and appended, waiting until the final “finish_reason” is received
already checked all chunks
