I thought the “CONTEXT WINDOW” meant (input prompt token + output prompt token). So I manually caculated the tokens for input before API Request, and truncated it some so that the output token doesn’t exceed the “CONTEXT WINDOW” for the model. But when I see the documentation, the “output token” i…

What does "CONTEXT WINDOW" mean in the documentation

elmstedt December 20, 2023, 10:47am 2

The context window includes all input, output, and control tokens.

1 Like