Hi systems
If your context is very long
A trick that works very well
Cut it in pieces of 3000 tokens
Then ask api to extract main informations as 10 bullet points
Then by aggregation of those bullets points
You will have a shortest context
Then when creating prompt
In system in prompt never speaks about context
Ask it to answer the question using the information provided like this
System prompt:
You are …
Your mission is to answer questions using data i provide you
If the answer is not in the data, don’t provide any answer
If the answer is in data give the answer without any comment
The format of your response should be an xml format:
<answer_in_data></answer_in_data> yes if you can answer with data provided, no if you can’t
<the_answer></the_answer> the answer without comment if you can answer, empty if you can’t answer
User_prompt:
Here is the question you should answer:
<the_question>
…
</the_question>
Using the following:
<provided_data>
…
</provided_data>
This prompt should work perfectly with gpt4 and 90% of the time with 3.5
Keep the xml tags
Note that xml tags name are using the same wording I’m using when introducing the mission and personnage
Very important to have good results
You can also (improve a little bit the results by 10% according to our benchmark) ask it to give a <note_on_10> for his answer and a short regarding quality of its answer but it will increase your costs
you will get an xml answer easy to parse
Knowing exactly when you have a correct answer.
Works perfectly for us (we have other tricks but those one should help)
Have a good day