Questions about the Output Token limit

API
1

hello, I have a question about the token limit. Since more and more models can now put tokens into context, I ask myself why the output tokens remain limited?

How does the output token limit come about? I always thought the tokens were counted like this: input+output.

But that’s apparently not the case, why is that the case, what is the limiting factor?

If it’s the computing power, then you could always generate 8k tokens and continue with the “Continue” button. But it doesn’t seem to exist anymore?

2

This site has some information on how tokens work.

3

Thank you very much for the link, unfortunately I had already gotten that far, but I still don’t understand why the output is limited if tokens can be seen as a whole. As already explained in my initial question, I have many questions about this.

1 Like