Should I put RAG data to additional_instructions for assistant right?

I am building chat support assistant. As global instruction in setup I put basic info about my product. However when user sends particular questions I first extract RAG context from my product knowledge base and I want to provide it to assiostant as addional instructions. As far as I understand, I should use additional_instructions parameter to avoid overriding (=erasing?) main instructions, if I would use instructions parameter per run?

One more question - how would I use additional_messages parameter ?https://platform.openai.com/docs/api-reference/runs/createRun#runs-createrun-additional_messages

1 Like

Additional messages can be a substitute for a separate API call that alone puts the new user message into a thread. Remember that user messages are permanently added, and contemplating a series of them isn’t a trained behavior.

You have a good idea for the use of additional instructions. This is simply more text appended within the system message, and is limited by the platform to 32k characters.

I would frame the RAG text as its own document to increase AI understanding. You can prepend a “header” such as:

Here is more knowledge injection, automatically retrieved from documentation, which may assist in answering the latest user question.

Chunk 1: {document}

3 Likes

can you please explain what do you mean here by “permanently added” and “contemplating a series” in this contextn as well. Thank you!

1 Like

A user message, even if added as a series of user messages that might have some RAG data before the question that the user asked, are part of the permanent conversation of a thread: they cannot be removed the next turn upon new RAG, and appear to be something that the user said in the past.

OpenAI has trained Chat AI models on conversations that take place as user/assistant turns. User/user/assistant/assistant or some odd variety like that is not “expected”, and the AI would have to infer what’s going on, with no clear answer of what it might think of the previous user inputs, whether to combine them, or whether to consider the older message something that happened in the past to be ignored.

3 Likes