I thought the “CONTEXT WINDOW” meant (input prompt token + output prompt token). So I manually caculated the tokens for input before API Request, and truncated it some so that the output token doesn’t exceed the “CONTEXT WINDOW” for the model. But when I see the documentation, the “output token” i…

Oh, thanks. Then the maximum (input + control) tokens for gpt-4-1106-preview would be 128000 - 4096 = 123904 tokens as I understand. Is this correct?

[image] soseol3191: 128000 - 4096 = 12396 123904 Give or take. Each message also has some miscellaneous tokens associated with the start and end of the message and who the message is from and whatnot. It’s on the order of 4 tokens per message. Typically, you’re going to have a bad time whe…

What does "CONTEXT WINDOW" mean in the documentation

API

anon22939549 December 20, 2023, 10:47am 2

The context window includes all input, output, and control tokens.

Topic		Replies	Views
What is the context window of gpt 4 API gpt-4 , chatgpt , api	4	17295	June 12, 2024
Number of context windows for GPT4V Community gpt-4	2	1995	January 26, 2024
GPT-4o context window confusion API gpt-4 , api , error , gpt-4o	5	61623	August 4, 2024
Regarding max input tokens of gpt-4o-2024-08-06 API gpt-4	3	4629	December 2, 2024
Context window size for the babbage-002 model Documentation question , fine-tuning , api	3	490	June 15, 2024

What does "CONTEXT WINDOW" mean in the documentation

Related topics