Impossible to generate texts of more than 600 words

API
1

Hello everyone,

I’m using the GPT-4 8k API for my application. There’s a big mystery surrounding the actual number of maximum theoretical tokens and those actually used.
When I write articles via my app, I struggle to obtain more than 600 words. And even then, most of the time, the responses don’t exceed 450 words.
My prompt doesn’t exceed 200 tokens, and yet the responses are still short.
So I’ve changed my prompt strategies, detailing each step it needs to take to reach the goal. He understands that he has to write a longer article, but he cuts off the replys with “to be continued…” or “there aren’t enough characters” even though he’s only written 400 words! How can I run out of tokens with only 400 words?

At most, prompt + reply don’t exceed 1000 tokens.
Of course- I’ve tried every setting with max-tokens.
Am I missing something or do we really not have 8K tokens?

Thanks in advance

1 Like
2

Hi @Ailogik ,

Have you thought about the quality of long text generated by GPT in one shot?.. Personally I find it disappointing and it produces far better results when you generate long articles in chunks of 3-5 paragraphs based on full outline and thoughts scheme for the chunk.

Here is the approach I take to not even run into a problem of text generated being to short:

1 pick up a topic to write about
2 brainstorm the concerned audience, article benefits to them, the goal of the article and tone
3 based on the above, get the outline
4 for each item in outline, make a draft of what needs to be said and how pieces are connected
5 for each of the draft, provide outline and the draft as context to the prompt to generate the section
6 combine the sections and run another prompt to improve the whole article

3 Likes
3

Thank you for your reply and for the advice. But that’s not the point. The question is why the api is unable to provide more output when we’re supposed to have 8k. As I say, the request barely uses 1000 tokens and it tells me “to be continued” or that it doesn’t have enough characters.

1 Like
4

What’s your prompt? We might be able to help if we saw it…

5
2 Likes