From the Docs: How it works The model then decides when to retrieve content based on the user Messages. The Assistants API automatically chooses between two retrieval techniques: it either passes the file content in the prompt for short documents, or performs a vector search for longer documents …

Understanding the current Assistant Retrieval process

CradleToCradle November 16, 2023, 2:15pm 4

Right now its very expensive to use when its not doing a vector-db retrieval. I’ve got a chatbot and did just a few messages today and used over 65k tokens…

1 Like

Strategy Recommendation for "Custom Code Generation GPT" through API

Topic		Replies	Views
New "Assistants" API a potential replacement for low level "RAG" style content generation? API	9	8293	March 4, 2024
Assistant API / costs / where do I find my token consumtions in assistants\|messages\|threads API	4	1893	December 14, 2023
Consequences of Assistants API for LangChain and vector database API assistants	5	3427	November 15, 2023
Did assistant api kill manual RAG with vector databases? API	8	5997	December 18, 2023
How does the knowledge of custom GPT actually work Documentation chatgpt	7	15163	December 1, 2023

Understanding the current Assistant Retrieval process

Related topics