How fast are custom GPTs at accessing uploaded knowledge store documents?

I have an Enterprise level client who wants to train an LLM to handle a few aspects of their customer relations.

It seems fine-tuning is limited to =< GPT 3.5 turbo. What are some alternatives to fine-tuning if I want to use the latest GPT? I know with gpt-4-1106-preview, it has a larger context size than gpt-4 and the ability to upload documents to a “knowledge store,” and I know of how I can create a specialized “GPT” in OpenAI’s web interface, give it that knowledge store of documents, and then access it via API. How good are these custom GPTs in terms of speed of accessing their knowledge store? How expensive to use? Is it even the path to go? Ideas?

Custom GPTs are exclusive to the ChatGPT web client and mobile app and can’t be called via API, but they can call your API. Assistants are the API equivalent, but it’s up to you to wire up some of the capabilities that are out of the box with ChatGPT.

The knowledge you can upload to them is very useful, but they don’t keep it in their context. If you upload a secret.txt with a single word, they need to read the file to see the content for every new conversation. They know of the file, but they don’t “keep it in their head”. The way I use this feature is to upload swagger/openapi definitions or large data structures, and it will load it whenever I ask it to make a serializer function for a particular type.

2 Likes

Thanks! You gave me the answers in a way I could understand! I’m a python coder who has worked a few months on upgrading my chatbot to use GPT-3.5 turbo then GPT-4, then GPT-4-preview; https://EmpathyBot.net. But not really sure which path to take to build this system for the client (a car dealership) that will pull it’s knowledge-base from (a) some training I give it, (b) analysis of customer complaint emails, (c) data from constantly changing inventory, (d) data from customer service record, and more. And as far as interface, (a) a chatbot on their site that they want to be empathetic (my specialty, but can’t be done well with GPT-3-turbo), (b) One that reads emails from customers and reply, and (c) an internal one for management or others to analyze various of the data mentioned above. I feel like I’m floundering without a clear direction. Ideas?

It would help if I understood what you meant by “The way I use this feature is to upload swagger/openapi definitions or large data structures, and it will load it whenever I ask it to make a serializer function for a particular type.”

One thing to add is that fine-tuning for GPT 4 is currently in limited public beta meaning only selected customers with a track record of fine-tuning OpenAI models got access to this highly advanced feature.
It likely makes sense for you to dig into the existing knowledgebase and create a solution around RAG and maybe a fine-tuned 3.5 model.

1 Like