# Clarification for max\_tokens **URL:** \ **Category:** API\ **Tags:** codex\ **Created:** [July 23, 2022, 9:36pm UTC](https://community.openai.com/t/clarification-for-max-tokens/19576 "2022-07-23T21:36:57Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![nashid.noor](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/nashid.noor/32/10976_2.png) [@nashid.noor](https://community.openai.com/u/nashid.noor)\ **Post date:** [July 23, 2022, 9:36pm UTC](https://community.openai.com/t/clarification-for-max-tokens/19576/1 "2022-07-23T21:36:58Z")
My interpretation for `max_tokens` is it specifies the upper-bound on the length of the generated code. However, the documentation is confusing. I am referring to the official API documentation [OpenAI API](https://beta.openai.com/docs/api-reference/completions/create#completions/create-prompt) ```auto The maximum number of [tokens](https://beta.openai.com/tokenizer) to generate in the completion. The token count of your prompt plus `max_tokens` cannot exceed the model's context length. Most models have a context length of 2048 tokens (except for the newest models, which support 4096). ``` So at first documentation mention the maximum number of tokens to generate in the completion. But then it states it is `token counts in the prompt` + `completion` \< `4000`. I mentioned `4000` as it is the maximum token limit for davinci model. So what is it? 1. is it the maximum token that would be generated during completion? OR 2. `token counts in the prompt` + ``completion` \< 4000 ---
**Author:** ![overbeck.christopher](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/overbeck.christopher/32/7035_2.png) [@overbeck.christopher](https://community.openai.com/u/overbeck.christopher)\ **Post date:** [August 30, 2022, 5:16am UTC](https://community.openai.com/t/clarification-for-max-tokens/19576/2 "2022-08-30T05:16:47Z")
1. `token counts in the prompt` + ``completion` \< 4000 ---
**Author:** ![kathyh](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/kathyh/32/24106_2.png) [@kathyh](https://community.openai.com/u/kathyh)\ **Post date:** [March 9, 2023, 5:08pm UTC](https://community.openai.com/t/clarification-for-max-tokens/19576/3 "2023-03-09T17:08:05Z")
I’m going with @overbeck.christopher here and staying on the conservative side the wording in context as original poster @nashid.noor pointed out remains confusing and leaves me questioning: Should I add my set `max_tokens` to the token count of my prompt to arrive at a number no larger than the limit of the model I’m using? These would be different numbers, but again I’ll work with the conservative approach for now. ---
**Author:** ![sps](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/sps/32/7736_2.png) [@sps](https://community.openai.com/u/sps)\ **Post date:** [March 9, 2023, 5:24pm UTC](https://community.openai.com/t/clarification-for-max-tokens/19576/4 "2023-03-09T17:24:51Z")
Hi @nashid.noor, @overbeck.christopher and @kathyh > [@Question regarding max\_tokens](https://community.openai.com/t/question-regarding-max-tokens/16259/2): > > Hi @florianwalther It completely depends on the prompt. Here’s the definition of max\_tokens in [API Reference](https://beta.openai.com/docs/api-reference/completions/create#completions/create-max_tokens): The maximum number of [tokens](https://beta.openai.com/tokenizer) to generate in the completion. The token count of your prompt plus max\_tokens cannot exceed the model’s context length. Most models have a context length of 2048 tokens (except for the newest models, which support 4096). Every model has a context length. It cannot be exceeded. As I shared above `max_tokens` only specifies the max number of tokens to generate in the completion, it is not necessarily the amount that will get generated. However, if the sum of tokens in prompt + `max_tokens` exceeds the context length of the model, the request will be considered invalid and you’ll get a 400. e.g `This model's maximum context length is 4096 tokens. However, you requested 4157 tokens (62 in the messages, 4095 in the completion). Please reduce the length of the messages or completion.` ---
**Author:** ![kathyh](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/kathyh/32/24106_2.png) [@kathyh](https://community.openai.com/u/kathyh)\ **Post date:** [March 9, 2023, 5:30pm UTC](https://community.openai.com/t/clarification-for-max-tokens/19576/5 "2023-03-09T17:30:40Z")
> [@sps](#): > > However, if the sum of tokens in prompt + `max_tokens` exceeds the context length of the model, the request will be considered invalid and you’ll get a 400. Thank you for answering my question in relation to how it plays out with each model’s context length. This is super helpful to understand the order of operations happening and when I could actually hit an error from mishandling these (my brain works backward seeing these terms in how they behave I guess). ---
**Author:** ![10basetom](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/10basetom/32/43457_2.png) [@10basetom](https://community.openai.com/u/10basetom)\ **Post date:** [August 13, 2023, 1:10am UTC](https://community.openai.com/t/clarification-for-max-tokens/19576/6 "2023-08-13T01:10:30Z")
Another point of confusion is `max_tokens` defaults to `16` – has anyone confirmed this? I haven’t used the API, but the ChatGPT website completions can be longer than 16 tokens. ---
**Author:** ![\_j](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/_j/32/766292_2.png) [@\_j](https://community.openai.com/u/_j)\ **Post date:** [August 13, 2023, 1:21am UTC](https://community.openai.com/t/clarification-for-max-tokens/19576/7 "2023-08-13T01:21:34Z")
> [@10basetom](#): > > `max_tokens` defaults to `16` That is only for completions endpoints, which makes setting the `max_tokens` value essentially required. For chat completion endpoint, you can simply not specify a max\_token value, and then all the remaining completion space not used by input can be used for forming a response, without needing careful tedious token-counting calculation to try to get close. Reminder, max\_tokens is a reservation of the model’s context length that is exclusively for forming your answer, as well as setting a limit to how much comes back. ---
**Author:** ![jagandevaki1](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/jagandevaki1/32/146604_2.png) [@jagandevaki1](https://community.openai.com/u/jagandevaki1)\ **Post date:** [October 3, 2023, 9:18am UTC](https://community.openai.com/t/clarification-for-max-tokens/19576/8 "2023-10-03T09:18:39Z")
> [@sps](#): > > ed 4157 `max_tokens only specifies the max number of tokens to generate in the completion` , can you explain what is max number of tokens to generate in completion? here what does completion mean? does it mean the response generated by the llm? ---
**Author:** ![curt.kennedy](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/curt.kennedy/32/709249_2.png) [@curt.kennedy](https://community.openai.com/u/curt.kennedy)\ **Post date:** [October 7, 2023, 11:28pm UTC](https://community.openai.com/t/clarification-for-max-tokens/19576/9 "2023-10-07T23:28:51Z")
> [@jagandevaki1](#): > > what does completion mean? does it mean the response generated by the llm? Yes, a completion is the response from the LLM. The word “completion” comes from the original models that would return the most probable completion text for your input text. So basically, the _autocompletion_. ---
**Author:** ![\_j](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/_j/32/766292_2.png) [@\_j](https://community.openai.com/u/_j)\ **Post date:** [October 8, 2023, 12:22am UTC](https://community.openai.com/t/clarification-for-max-tokens/19576/10 "2023-10-08T00:22:28Z")
Here is max tokens of 6, with a completion model. It does a very advanced version of writing what comes next: ![image](https://us1.discourse-cdn.com/openai1/original/3X/c/e/ce67cb6676ffaf7b519cc45357e5c6f6d9a88a48.png) The colored text is AI’s six tokens of **completion** output after my un-highlighted writing **prompt**. Because this “completion” is so talented and versatile, we can give other writing formats for it to complete: ![image](https://us1.discourse-cdn.com/openai1/original/3X/f/4/f4de32301af075718b6ff913fa0218c8de31e7c9.png) With only six tokens, we didn’t get much text, however, I can have it “complete” what it was writing again for another six tokens: > I like the name “Ava.” It is simple, So the `max_tokens` value will cut off the AI’s output if you don’t set it large enough. The AI doesn’t know what this setting is. The chat endpoint puts all the “human” and “AI” banter into containers, and the model has been trained to perform in a conversation setting. ---
**Author:** ![EricGT](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/ericgt/32/20571_2.png) [@EricGT](https://community.openai.com/u/EricGT)\ **Post date:** [December 12, 2023, 6:30pm UTC](https://community.openai.com/t/clarification-for-max-tokens/19576/11 "2023-12-12T18:30:11Z")
Closed on December 12, 2023, 6:30pm UTC