Hi folks, I’m using GPT-4 api to do RAG and chat with PDFs, I’ve tried the langchain way and also the openai assistant way. In the langchain way, the implementation involves dividing the content into multiple chunks, and use openai embeddings to embed the question and the context, and generate the answer using FAISS, and langchain chat with openai functionalities. However, I often encounter the problem that the API tells me it cannot find appropriate answers, while if I am using the chatgpt UI, it can tell me the answer. There is a large performance gap between using the API and using the chatgpt UI. The UI is very intuitive while there are tons of things to control with the API. Is there an easy solution for matching the API’s performance with the UI’s performance?
1 Like
Try different chunking algorithms and optimizing the retrieval by changing your threshold or your top-K, it might give better results
Thanks, I’m actually curious why OpenAI did not open this part as part of their API? Or is that there is no solution that fits all? At least have something ready that achieves the same level of performance that we have seen in the UI would be desired. It will surely make life easier for developers, and I’m more than happy to pay for the cost.