Using chatgpt for a creative application. I’ve got 3-5 LLM services, and they all talk to one user. Was thinking of merging their chat histories together, so they can see the context of what happened when a user talked to a different LLM and got x and y back, because its relevant to their tasking.
Was wondering, is this a bad idea or could it work? What I am worried about is that the other LLMs will hallucinate or get confused on what their output should be because they see output from a different LLM that contains assistant messages like tool calls they can’t ever call, or output they should never generate, but that output itself is good context for the other LLM to be aware of when those llms craft their own outputs for their specialized task.
Any ideas? Could I move this history into the system prompt, so the other LLMs dont think its what they should be generating themselves?