I always go back to this post, but it would be nice if it was formalized in some way: How to stop models returning "preachy" conclusions - #38 by Diet
The idea is that we know that recency is important.
When generating any text, if the model were allowed to (and some people, too), it would just keep rambling on in the same style, fortify that style, and eventually converge on infinitely printing a single token (assuming we we turned off frequency bias).
But we can use low frequency patterns (like a format, e.g.: json, xml, markdown headers) to break the generation out of a high frequency pattern (e.g.: generating a paragraph).
And these low frequency patterns/formats can be directed (with like a schema, or a table of contents, for example) - to draw model attention away from the recent, and instead to a specific needle deeper in the context.
Ok now that I put it like this it sounds kind of obvious: use a table of content or a check list to make sure all topics get covered. duh.
But I think the nuance is here that different formats have different strengths. A weak format is no match against trained behavior.
Does that make sense? I’m hoping you can formulate all this better than I can.
Oh yeah, this is more for situations where you can’t afford to give exhaustive examples at the risk of distracting the model from the content. This is more about getting the model to show itself how to follow instructions. reinforce while generating. sorta. 