I only need the function call functionality of the OpenAI API, so I want to reduce the maxTokens count when sending requests to the API. My objective is to obtain the name of the called function in the response. Do I need to make any changes to the maxTokens field or take any other steps to achieve this? Because generated content is additional cost for me and i want to minimize it.
2 Likes
Tell the AI to be succinct, concise, any of those kinds of words and possibly add “Only return the function call, do not output anything else” You may need to experiment with exact wording, but that generally gets you a compact reply.
1 Like
You really helped a lot by telling: ask for ai 
I am just asking that is it good practice to
reduce max_tokens’ count for my goal
In my experience the max_tokens variable will not change the size of the generated reply, it will only truncate it if it is to large.
Is your idea to try and set the token size to 1? and try and get back a valid function name?
it doesnt work with 1 token count ofc but i thing that there should be some particular value for max_tokens for acceptable cost.
well, you could try and handle the data yourself, create a socket and send strings to and from the endpoint, that way you could monitor the raw text being returned and extract the function name with the minimum of tokens, fo course that would require a more low level approach and would get more technical. In my Profile is a link to my github where I have a C++ library for arduino framework code to connect to the API endpoints, it contains a low level implementation of the comms, you may find it of some use.
1 Like