I have uploaded a 34-page document as a knowledge base to the assistant, and I have found that it requires a minimum of tens of thousands of tokens for the context when making API calls.
So, what is your question?
It seems like the question is “wtf?!” and we can close the topic 
My question is that when using the Assistants API with a knowledge base, the number of tokens required for the context is in the tens of thousands, and this is only for the first question in a session. The number of tokens will multiply for subsequent questions, making the cost too high. I hope this can be improved.
1 Like
Going quadratic is by design. Everybody has free research grants to run AI that can call itself with little limitations, and have $20 “chat with my PDF” sessions you were all asking for. Or so one would think with Assistants. “Beta” should go back to “alpha” for some re-engineering.
At the same time, you want knowledge from within 5MB or 50MB? All that text will need some kind of search indexing before some can be used to to fill the model for answering regardless, and OpenAI is going to make that AI-powered for highest effect.
What I meant to express is that the initial call to the Assistants API for the knowledge base context already requires tens of thousands of tokens, which is generally unaffordable for most people. Additionally, the GPT-4-1106-preview model has a maximum limit of 500,000 tokens per day, which imposes constraints on the number of questions that can be asked each day.