Hi! Anyone know if OpenAI has plans to support the retrieval tool in the Chat API (not the Assistants API)? From the docs:
Currently, only functions are supported as a tool.
Use of the word “currently” leads me to believe this may be on the roadmap.
1 Like
I’m happy to tell you that it’s already a thing. You just have to build all of it unfortunately. OpenAI cookbook has had a chat completions retrieval repo and tutorial for a while, and there are a ton of public repos from other devs, but I wouldn’t try 
I’ll explain…
You need to set up and query against an indexed vector store like Pinecone, or find an existing retrieval API and create a function that chat.completions can call. There’s a lot more to this and I’m lazy.
I gotta tell you though, the main point of the Assistants API is to lower the entry level skills needed to do RAG. The API can now manage and do retrieval on the OpenAI side. They also help manage and host all your conversations with the Assistant.
The chat completions API used for “chat” features is outdated, they really need to rename it now or release a blog.
Chat completions should really now just be used in your backend for server-side AI decision making. In other words, let’s say you want to have your Assistant’s work checked before it goes out to the user. You could code a bunch of complicated gates and checks, or you could have a gpt-3.5-turbo chat one-off analyze the response, edit it and then sending.
Or you might be trying to process a massive amount of data. You could have 3.5 do summaries in chunks and then have GPT-4 assistant use them to craft the end response.
It’s now best used as a single shot, dynamic logic module.
1 Like