Hello OpenAI team,
I have noticed that if max_prompt_tokens is set low like ~500 some very strange hallucinations start happening with code interpreter and chat responses. probably due to messages being dropped due to tokens taken up by tool definitons.
If the max_prompt_tokens is set to ~10.000 or less some very strange hallucinations happen with file search.
I just wanted to let you know.
Thank you always for great models and the API!
Thanks for highlighting this! We’ll add some documentation around reasonable numbers to use for max_prompt_tokens when using tools.
Hi there. Have the same issue. I set max_prompt_tokens to 5 thousand with file search and get heavy hallucinations! Model started to create random answers with mentioning some strange document references… How to find the right number to strict input tokens? For example I have document with template answers, in file citation it takes only one paragraph from it, but read 10000 tokens and more as it read all file. How to optimise this file search?
We’ve added some documentation around this. When using file_search, I recommend keeping the value at or above 20,000. See https://platform.openai.com/docs/assistants/how-it-works/max-completion-and-max-prompt-tokens
I am seeing an error of the run to be incomplete with the reason:
incomplete_details: {“reason”:“max_prompt_tokens”}
Even though the “max_completion_tokens”:50000 & “max_prompt_tokens”:50000. There is no file search involved, just simple prompting.
If it is of any use the threadId is thread_ZMTTCmqnTr4MUmtO79rZjdg1 and runId is run_PKlRJ34tTiJ3rQruUwYFwNXV
This message has more details
