Incomplete or truncate result

API
1

Hello, I am using the following API call but the result is incomplete or truncated. I get results like this:

#NorthKorea-linked #LazarusGroup transfers $63.4M in #Ethereum from #2022HarmonyBridge hack to #crypto exchanges. Funds frozen, 124 $BTC recovered. Attack is part of a larger #crypto theft operation totaling …

API call:

    const response = await this.openai.createCompletion({
      model: 'text-davinci-003',
      prompt:
        'Create an objective summary news from this article with less than 260 characters including spaces:\n' +
        text +
        '\n',
      temperature: 0.7,
      max_tokens: 65,
      top_p: 1.0,
      frequency_penalty: 0.0,
      presence_penalty: 0.0
    });```

How can I improve it?
2

I think I will be right in saying that your max_tokens is way to low for what you are doing.

You need tokens that accommodate input and output. Tokens equate crudely to 1 per every 4 letters.

A high max_tokens will not cause them to be consumed unless they are used. And they are a generally a fraction of a cent per 1000.

2 Likes
3

Thanks for the answer. do you know how I can get the result in more than 260 characters? The result is bigger.

4

Just as @paul.armstrong said: set a higher max_tokens.

    const response = await this.openai.createCompletion({
      model: 'text-davinci-003',
      prompt:
        'Create an objective summary news from this article with less than 260 characters including spaces:\n' +
        text +
        '\n',
      temperature: 0.7,
      max_tokens: 1000,
      top_p: 1.0,
      frequency_penalty: 0.0,
      presence_penalty: 0.0
    });```
5

Sorry, the result I what to be less than 260. I wrote “more than” by mistake.

6

Something like this?

[TASK: Create a short, objective summary from this article.\nLENGTH: less than 260 characters including spaces.]

7

I’m having the exact same issue. Please let me know if you got the solution to this issue.

8

Hi @sukant.bizwatch
You can Add the instructions and say that these instructions are to be Adhered while generating responses.

An example would be like this

You are a paraphrasing bot. You have been assigned to create paraphrases of the content given below. You will get set of instructions under the `Instructions` tag. You have to adhere to the instructions  while creating paraphrases.  The content which is to be paraphrased will come under `Content` Tag.  

 Instructions: 
- Tone should be witty. 
- No sentences should be repeated. -
 Output should always be in complete json structure like this    
{    1 : this is paraphrase example 1 . ,   
     2 : this is shorter paraphrase example  . ,    
     3 : this is longer paraphrase example , should be in 50 words  .    }
9

The solution to the “incomplete or truncated result” that you likely discovered as a search term in order to resurrect this six-month old conversation:

  • The poster had set the max_tokens parameter too low.

Not specifying max_tokens will also give a very small output with the default value.

10

Hi
Actually, I’m trying something similar. I’m using the following API call:

import os
import openai

openai.api_key = os.getenv("OPENAI_API_KEY")

response = openai.ChatCompletion.create(
  model="gpt-3.5-turbo",
  messages=[
    {
      "role": "system",
      "content": "You are __<PersonA>__.  Abide by the below rules while impersonating __<PersonA>__:\n\nInstructions:\n- Embody her vision and self-expression.\n- Maintain bold and refined demeanor.\n- Output should always be a complete sentence\n- Ensure that each reply offers a complete concept while following a word limit of 40 words,  10% more or less."
    }
  ],
  temperature=1,
  max_tokens=100,
  top_p=1,
  frequency_penalty=0,
  presence_penalty=0
)

But the responses I’m getting are either incomplete or truncated.

For ex, if I’m asking “Tell me a story in 100 words”, I want the response to be in approximately 40 words and in complete sentences as instructed in the system prompt, but the responses I’m getting are between 70 - 90 words.

How can I improve the prompt to get the desired output?

11

You’re confusing it… you’re saying you want 100 words but in your mind you want it under 100 words or even better less than 100 words or even better just say around 50 words…

12

This parameter sets the maximum response you can get back:

max_tokens=100

Try 1000-1500, unless you have a specific reason for that setting.

Specifying 100 tokens means about 60-70 words. For example, this response to you, if written by the AI instead of me typing it into a token-counting interface to see how long it needs to be in order to consume a certain number of tokens, comes to 101 tokens in total.

1 Like
13

Point being, I want it to follow the system prompt even if it is asked to generate a response with 40+ words. Is that not possible?

14

I want the responses below or approximately of 70 words. Like for “Tell me a story in 100 words”, can it not tell me a story in 40 or so words following the system prompt only. I won’t mind even if it starts it response with "I’m unable to tell you a story in 100 words but here is a small story in a few words …(something like this)?

Another thing, even if I take out the constraint of 100 words, some responses I’m getting are either incomplete or truncated. How do I avoid this?

Since I’m new to this, I would really appreciate some help.

15

The AI can’t count well or estimate how it should use language while it is writing to meet a particular goal, and also doesn’t know about maximum token setting.

One should guide its creation by other language, like “three sentences”, “a brief paragraph”, an essay, a parable, etc:

Tell me a one-paragraph story in three sentences about a schoolboy’s mischief.
Amidst the classroom’s hushed routine, mischievous Jake concocted a plan to swap the teacher’s chalk with a stick of white chocolate. As the unsuspecting teacher picked up the “chalk” to write on the board, a playful grin spread across Jake’s face, but his laughter gave him away, earning him a scolding and the reluctant admiration of his classmates for his audacious escapade. – 84 tokens

2 Likes
16

Specifying no. of sentences helped a lot!!
Thank you so much!

3 Likes