When i set max_tokens to 200 it just give me empty response randomly. Not for each request. I even tried using stop but still the error remains.
The max_tokens parameter, or now max_completion_tokens, has no effect on what the AI generates. It only stops the AI from generating after it reaches the maximum length you specify, to limit your cost of unexpected large outputs.
To be able to offer any help, knowing what “it” is that you are talking about is of high importance. For example, if you limit a structured output JSON by length, you’ll get an error instead of a parsed output.
Following is my payload.
response = self.client_openai.chat.completions.create(
model=self.openai_model_name,
messages=messages,
temperature=0.5,
timeout=30,
seed=1,
max_tokens=200,
)
Still I’m getting empty string randomly