I have uploaded few pdf files to the assistant playground, each file contains 200~pages, each page contain a reference.
I prompt the assistant to get me all references (in the page) and short description of its, but I always get less than max num of result which is no higher than 50.
I prompt it to iterate through all chunks in vector store (not sure how it is chunked or the chunk size) but still nothing changes.
how do I get it to reply with all the references and their description in knowledge base?
The vector store of the Assistants playground only operates by a semantic search, where the AI writes a query, and the highest ranked results are returned.
If you need to reference pages directly, you’ll need to provide the plain text directly in messages, and for a job that works its way through chunks of your own extracted document text, you likely won’t need the conversation history of Assistants, and can use the Chat Completions endpoint instead.
for some reason this system instruction worked only for getting all pages references if you were asked to get all page references and their short summary, loop through the entire database chunk by chunk and return the requested data in an excel file
but it is failing to get the description/summary of each page, it is getting some content not even in the knowledge base and it kept streaming for almosst 5 minutes. for this I know there are 207 pages, it returned 207 page references and over 1000 summaries