I’ve been testing the File Search Assistants API, and it’s neat but even an unused file search tool adds 551 tokens to the initial request.
Is this an error or is the hidden prompt for the file search tool really 551 tokens long?!
Yep, it’s really that long. Lots of text for the AI about how to use it and emit langauge and how to write citations into responses.
Were it merely a search function using function-calling, which it basically is, use would be pretrained and obvious to the AI.
You can now trim some of the fat with a new API parameter to remove parallel tool calls and the text telling the AI how to use it.