GPT-4o-mini max token 16,384

API
,
1

Hello everyone,

I’m currently working on integrating the GPT-4o-mini model into my project, and I need to generate text outputs with a length of up to 12,000 tokens. However, I’m facing an issue where the output gets cut off before reaching the desired length.

Here’s a snippet of my code:

const generateText = async (prompt, maxTokens = 16000, model = ‘gpt-4o-mini-2024-07-18’) => {
try {
const response = await openai.chat.completions.create({
model,
messages: [{ role: ‘user’, content: prompt }],
max_tokens: maxTokens,
});

const tokensUsed = response.usage?.total_tokens || 0;
trackTokens(tokensUsed);

return {
  response: response.choices[0].message.content.trim(),
  tokensUsed,
};

} catch (error) {
logger.log(‘error’, OpenAI API Error: ${error.message});
throw new AppError(OpenAI API call failed: ${error.message}, 500);
}
};

Despite setting maxTokens to 16,000, the response is still cut short, and I’m not achieving the 12,000 tokens I need.

Could anyone provide guidance on how to ensure the model produces the full output length? Any advice or suggestions would be greatly appreciated.

Thank you in advance for your help!

2

Welcome to the community!

Can you describe (or give an example) of how the response is cut short?

I don’t work with long responses because I think they’re unstable, but in the past models have been trained to figure out creative ways to cut responses short.

Do you get a token limit reached in your response end reason? or a stop token? Does it end gracefully at the end of a sentence/paragraph?

Meanwhile, a thing to keep in mind is that the maxTokens parameter is a limiter, it just cuts generation off. if you want the longest possible generation, you can simply leave it blank. the model doesn’t see it, and it doesn’t inform your output.

2 Likes
3

Thanks a lot, i took away maxtoken and it worked :pray:

and yes it was like the AI response have been cut, because the answers wasn’t completed like at the end of the anwser you find something like that.
Exemple “… integra” but i should be “… integration … (more text)”

3 Likes