Hello everybody.
I’m creating a chatbot for internal use that will respond based on PDF files converted to text.
However, these files are very large and I cannot pass all the content via API as it would consume many tokens and even be impossible in some cases.
I would like suggestions and some clarification on the best way to obtain the most relevant text to send as context via API.
Do I necessarily need to use a vector database?
Any tools to extract the most relevant text as context?
Any other tips will be very important, as I am a first-timer.
The least effort way you can do this right now is via the Assistants API. You can start now by creating your own Assistant, select a model to use (GPT-4-1106-preview, etc), add Instruction(system prompt), upload your PDF file (maximum file size is 512 MB), turn on the Retrieval from the Tools and start chatting. Just pay attention to the tokens you are consuming. You can select 3.5 models if you just want to get the hang of things before you try GPT-4 models. Also, files uploaded for Retrieval is free until 1/12 next year and Code Interpreter is free to use until 12/13 this month.
Thanks for your response. This seems very promising although the issue of token consumption seems like a problem, as I didn’t see how the developer could control it, I saw some here in the community reporting very high spending using the Assistant.