How did ChatGPT learn different text varieties/text types/genres?

Hi everyone, how did chatGPT learn to distinguish between different text varieties such as song lyrics, news, and research articles, for example? When you prompt it to produce a text from a particular variety or in a particular style, it shapes the output in a way that resembles the desired variety or style (to some extent). Were (at least some of) the the texts that it was trained on labeled with tags representing the general variety of the text? If not, how is that knowledge acquired or passed on to it?

Thank you ahead

Nobody knows nor has a complete answer. That is why people call these neural network architectures a “black box”.

LLMs are simply trained on huge amounts of textual data. some are tagged, some aren’t. Although, it’s not really that simple, and each company has their own technique for developing this.

The ability to identify and replicate patterns from wildly different kinds of linguistic data is simply a byproduct of the mathematical algorithms and model weights that underpin these neural networks.

1 Like