OpenAI parameter max_tokens minimum

API
1

Hello,

In OpenAI API, is there a minimum value for the max_tokens parameter?

Thank you in anticipation.

Martin

2

Yes, the maximum value is whatever the model size is, 4K, 8k, 16k, 32k, 128K minus your prompt size in tokens.

3

Hello,

The question is about a “minimum” max_tokens?

Regards,

Martin

4
Yes. 1 
Error: 400 {
  "error": {
    "message": "0 is less than the minimum of 1 - 'max_tokens'",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}

reproduce:

import requests
import json
import os

# Ensure you have your OpenAI API key set in the environment variables
openai_api_key = os.getenv("OPENAI_API_KEY")
if openai_api_key is None:
    raise ValueError("OpenAI API key is not set in environment variables.")

url = "https://api.openai.com/v1/chat/completions"

headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {openai_api_key}"
}

data = {
    "model": "gpt-3.5-turbo-1106",
    "max_tokens": 0,
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "Hello!"
        }
    ]
}

response = requests.post(url, headers=headers, json=data)

# Check if the request was successful
if response.status_code == 200:
    print("Response from OpenAI:", response.json())
    print('\n')
    print(response.json()['choices'][0]['message']['content'])
else:
    print("Error:", response.status_code, response.text)
1 Like
5

While you can specify 1 token and get 1 token back, which is good for single token answers, and also evaluating that token’s probability, there is another limit you’ll find: functions.

When the AI emits functions, it needs enough tokens to write to the tool recipient with a well-formed beginning of a response that the internals of the API can parse. Otherwise you’ll get a 500 server error. About 12 tokens depending on the tool and function name length.

1 Like