If I’m adding context to the conversation (that has been either determined by parallel summarization, function calling to remember specific facts, etc), does it matter if it is provided in the first system message, or if it’s injected into a later system message?
The model reacts a little bit human like in that regard.
You most probably read a longer text by taking the first sentences to get an idea about what it is about, then you scroll through it depending on how it is formatted and then you want to see the resoultion in the end to check if it is worth it to delve deeper into it.
So as far as I have read there is a huge difference between putting something in the middle or in the beginning or end of a greater context.
I bet you can test that when you put texts in different locations and ask the model to summarize the whole thing into one sentence or something.