I think I’ve seen ChatGPT write backticks for Markdown codeblocks before it would render into codeblocks. Either way, while LLMs can be told to output answers in Markdown with internal prompting, how exactly is that ability implemented? Is it like that, LLMs get training to answer in markdown (a fine-tune, though I don’t how fine-tuning works), or something else?
Presumably the training data has enough examples of markdown formatted code that it is a naturally emergent behaviour.
But I mean how does it know to always use markdown for answers though? Like write codeblocks for code. Are there any systems to ensure that or is it just trusting the model to do so, and it could theoretically output code without code blocks? (and possibly neglect markdown in other places where appropriate)
The markdown format is reinforced over and over in fine-tune training data (RLHF) by a machine learning reward model, to where the AI now can’t hardly live without it, and will use code blocks even when undesired.
Token sequences in predict token sequences to follow.
Earlier gpt-3.5-turbo-0301 model for example, you can train the AI yourself that it lives in an HTML render environment with prompting. gpt-3.5-turbo-1106 = now useless for being anything but ChatGPT.