Issue with max_tokens in case of using functions

I employ a function to instruct OpenAI’s chat completion API to structure the response, including source references. The formatted response appears as {"output": <response>, "sources": [.....]"} . However, I’ve noticed an issue when setting max_tokens = 100 . It seems that the API interprets it as the maximum token limit for <response> alone, not accounting for the entire string present in function_call.arguments . Consequently, I only receive 100 tokens in function_call.arguments , resulting in an incomplete string where the end is truncated. Anyone have had the same issue? How to solve?

The max_tokens is the amount of language generation you want to allow the AI before production of tokens is terminated. You are seeing the parameter operating correctly.

Yeah, well… this is not exactly my question…
When utilizing the chat completion function, it adds specific data into <response>, which is a portion of the total function_call.arguments. Given that this function is an integral component of the API, my expectation is that the max_tokens parameter would limit the entire response included in function_call.arguments, not just the <response> subset.
If all allocated tokens are consumed for populating <response>, none are left for the other components of the function’s output. This, in my perspective, seems to be an inconsistency in the API’s behavior, as it doesn’t align with the anticipation that max_tokens should govern the entire content of function_call.arguments.

The entire response within a function is ai-produced language. The json structure from your function specification is made of ai tokens.

Understand.
In my case, the json returned in function_call.arguments is truncated; therefore, it is ill-formed (without some text, closing quote, and curly brackets) and cannot be processed further. The reason appears to be that the max_tokens is applied on the , not the function_call.arguments. quote, and curly brackets) and cannot be processed further. The reason appears to be that the max_tokens is applied on the , not the function_call.arguments.