Gpt with own knowledge using vdbs

API
,
1

Hello everyone! I am trying to feed gpt-3.5 with my own “custom” knowledge base. But I found out that this is very expensive, because with every message the whole knowledge base has to be given again to the ai which if the knowledge base is really big might be a couple thousand tokens per message. I’ve looked into vdbs but i dont really get how i could use them in a chat completion api call. Could somebody help me out here? Thanks!

For context. This is my current api call data:

{
  "model": "gpt-3.5-turbo-0125", 
  "messages": [
     { "role": "system",  "content": knowledge },
     { "role": "user", "content": content }
  ]
}
2

Hi @ikonodim,

At 2000 tokens, the knowledge base isn’t particularly large, but I understand that this may result in some costs that increase with every successive turn.

I wrote a tutorial about how to use embeddings to retrieve relevant context for an AI assistant, which you may find helpful

Additionally, here’s the OpenAI Cookbook with a bunch of tutorials. At the bottom, you’ll find RAG implementations with some popular vector DBs.

I’d also recommend that you segregate the instructions and the context (knowledge) to ensure the model behavior remains appropriate even if the context isn’t found.

1 Like