Functions VS Tools - What is the difference?

API
, , , ,
8

Certainly. Do you expect “nobody knows how to make it work”?

Here is an example request that takes each stream chunk as they are received, and displays any content and adds any tool_call parts to a list:

c = client.chat.completions.with_raw_response.create(**params)
reply=""
tools=[]
for chunk in c.parse():
    # This gets content, which is clear text meant for a user
    if chunk.choices[0].delta.content:
        reply += chunk.choices[0].delta.content        # gather for chat history
        print(chunk.choices[0].delta.content, end="")  # your output method
    # This gets whole tool chunk objects, that need later assembly
    if chunk.choices[0].delta.tool_calls:
        tools += chunk.choices[0].delta.tool_calls     # gather ChoiceDeltaToolCall list chunks

What we have gathered delta parts in the “tools” list when they had that element in the json of the chunk.

At this point, we’ve now got a python list list of objects, (the library’s ChoiceDeltaToolCall). They look like this if we dump to dictionary:

for tool in tools:
    print(tool.model_dump())

collected from the streamed chunks:

{'index': 0, 'id': 'call_P0WmAn1FTpLTlJtz18a53z', 'function': {'arguments': '', 'name': 'get_random_float'}, 'type': 'function'}
{'index': 0, 'id': None, 'function': {'arguments': '{"', 'name': None}, 'type': None}
{'index': 0, 'id': None, 'function': {'arguments': 'range', 'name': None}, 'type': None}
{'index': 0, 'id': None, 'function': {'arguments': '_start', 'name': None}, 'type': None}
{'index': 0, 'id': None, 'function': {'arguments': '":', 'name': None}, 'type': None}
...

You can see then that we need to do a bit of parsing to put these back in the same type of standalone tool call object we’d get with non-streaming. Let’s use a function:

tools_obj = tool_list_to_tool_obj(tools)

Which I have written out before…and you can avoid the search by clicking.

1 Like