I am using the assistant to retrieve data from an external API. The amount of data that this API returns can vary. The assistant’s reply to the user must be 2000 characters or fewer. I have tried setting max_completion_tokens to a small number (I used 500) but got a incomplete_details: { reason: 'max_completion_tokens' } error. I also tried setting a limit in the system instructions by including Give your answer in 2000 characters or less but this also doesn’t change the length of the output. Any help here is greatly appreciated!
The AI doesn’t have an understanding of what you set max_completions_tokens to. That parameter is doing its job in cutting off the output where you set it.
Instructional language to the AI indeed need to be your method, along with ensuring those instructions are understood as the response format, or are what the user is specifically requesting as the task to be performed.
Information returned from a tool function will have a different quality of reproduction to it. If you call the tool “summarize_text”, the AI is going to assume the function did the job, not it.
If you are using functions and returning plain text of a document, that text can also have instructions, like “Here is an original file. You must never repeat back file contents wholesale. You now produce only a small encompassing summary of what has been returned below, maximum length three pararagraphs.”
That’s really helpful, thank you 