In lights of the recent announcement for function calls in the new GPT-4 release Can you please clarify if the function definitions consume tokens?
1 Like
Yes. It does. I just checked.
gpt-3.5-turbo-0613, 1 request
119 prompt + 15 completion = 134 tokens
In that sample, about half of 119 tokens for the prompt is the function call.
Even normal chat completion calls consume hidden tokens.
Does anyone know how to compute the number of tokens which would be used for the function definition, before calling the API?
There’s links to some libraries at bottom of page
Yes there are examples of how to use tiktoken to count the token usage for system and user messages, but not for function definitions.
EDIT: after some testing it looks close to just computing number of tokens for json encoded function definitions.