Hi, I am providing the users ability to create agents.
I want them to be able to give some good examples of sessions to the agent from a file.
What is the best way to do this given that assistant api is in beta and I need to use chat completion for my agents?
You don’t “need” to use Chat Completions, but they do give you more control at the cost of additional infrastructure of your own.
If using the latter you will need to upload the files to your local DB which might be a combination of extracting the text, images and metadata and then enriching one or more chunks of this data with an embedding vector which can the be used for search. You can use the Embedding API for this.
For retrieval you can add a search function that you let the LLM know about and it will call that function based on the user query.
You will need to implement a parser to detect that a function has been invoked by the LLM and then implement that function with local code.
Usually you’d take the embedding of the search term and look for chunks of your uploaded files that closely match (eg low cosine distance) your search term embedding.
Then you’d take a small set of close results and send them back as answer to the function, thereby enriching the prompt to the LLM with text from your files that are closely associated with the user query.
1 Like