Sort of. There’s nothing actually hardcoded, but one pattern in the data GPT was trained on is that lines of characters like that denote a sort of change in context. Remember that GPT is just giving you what it thinks the most likely next token is. It’s not just a pile of statistics, but that is a big part of it. If you don’t put something like that in then you might get issues where GPT starts writing more instructions instead excecuting them.
2 Likes