'50 words in one response' versus 'n=50 single-word responses', what is cheaper?

API
, , ,
1

Which approach is cheaper?

Approach 1:

#python
context = "Write 50 words in the format 'Word 1, Word 2, Word 3' (do not numerate)."
response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[
        {
            "role": "system",
            "content": context
        }
    ],
    max_tokens=500
)

Approach 2:

#python
context = "Write 1 word."
response = openai.ChatCompletion.create(
      model="gpt-3.5-turbo",
      messages=[
          {
              "role": "system",
              "content": context
          }
      ],
      max_tokens=10,
      n=50
  )

The first approach has a longer input since I need to specify the formatting method for the model. Without this, it might use a different separation method for the response, which would make data extraction more challenging. On the other hand, the second approach has a shorter input because I’m only asking for one word and I’m already getting 50 responses, each separated in the list. Based on what I’ve understood from ChatGPT, the second approach will likely be cheaper, as it’s not going to use input tokens for each of the 50 responses, but only once. I just want to make sure this is true and that there are no hidden fees for each of those 50 responses.

2

I did not understand the context of the question. the first one looks like will keep of looping, right. the next one also will loop but in each iteration it will create 50 completions with max 500 tokens + prompt tokens each. It seems to me like the next one will be costlier

3

they’re both pretty costly of you never break your loop :japanese_ogre:

1 Like
4

I’m so sorry, my bad.

Approach 1:

#python
context = "Write 50 words in the format 'Word 1, Word 2, Word 3' (do not numerate)."
response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[
        {
            "role": "system",
            "content": context
        }
    ],
    max_tokens=500
)

Approach 2:

#python
context = "Write 1 word."
response = openai.ChatCompletion.create(
      model="gpt-3.5-turbo",
      messages=[
          {
              "role": "system",
              "content": context
          }
      ],
      max_tokens=10,
      n=50
  )

The first approach has a longer input since I need to specify the formatting method for the model. Without this, it might use a different separation method for the response, which would make data extraction more challenging. On the other hand, the second approach has a shorter input because I’m only asking for one word and I’m already getting 50 responses, each separated in the list. Based on what I’ve understood from ChatGPT, the second approach will likely be cheaper, as it’s not going to use input tokens for each of the 50 responses, but only once. I just want to make sure this is true and that there are no hidden fees for each of those 50 responses.

5

My apologies, I copied the wrong piece. I’ve edited the initial post.

6

Your request may use up to num_tokens(prompt) + max_tokens * max(n, best_of) tokens, which will be billed at the per-engine rates outlined at the top of this page.

Based on this I think #1 would be more expensive since you are charged for the prompt each time, whereas #2 you are only charged once.

But give a try, make the first request, wait an hour with no requests, then make the 2nd request. Check the usage page and select today in the Daily breakdown and you can see how many total tokens they billed for.

7

An entire hour without GPT??? That’s insane!

Just kidding :smiley:
Thank you for fast response.

8

I could not control my curiosity as I thought #2 will be more costly. I ran both and looks like #2 indeed is using more tokens

#1

{
“choices”: [
{
“finish_reason”: “stop”,
“index”: 0,
“message”: {
“content”: “Apple, banana, orange, taxi, plane, train, happy, sad, excited, tired, busy, lazy, ocean, mountain, city, rural, technology, art, literature, politics, history, science, mathematics, music, film, food, drink, summer, winter, autumn, spring, love, hate, friendship, family, education, career, success, failure, health, wealth, happiness, meditation, yoga, exercise, sleep, adventure, relaxation, challenge.”,
“role”: “assistant”
}
}
],
“created”: 1685131038,
“id”: “chatcmpl-7KXqoVt3dpOdz0uamzguUzSuAkMRO”,
“model”: “gpt-3.5-turbo-0301”,
“object”: “chat.completion”,
“usage”: {
“completion_tokens”: 98,
“prompt_tokens”: 34,
“total_tokens”: 132
}
}

#2

{
“choices”: [
{
“finish_reason”: “stop”,
“index”: 0,
“message”: {
“content”: “Flower.”,
“role”: “assistant”
}
},
{
“finish_reason”: “length”,
“index”: 1,
“message”: {
“content”: “Hello. \n\nI’m sorry, I cannot complete”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 2,
“message”: {
“content”: “Pineapple.”,
“role”: “assistant”
}
},
{
“finish_reason”: “length”,
“index”: 3,
“message”: {
“content”: “Supercalifragilisticexpialid”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 4,
“message”: {
“content”: “Prompt.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 5,
“message”: {
“content”: “Cookie.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 6,
“message”: {
“content”: “Cactus.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 7,
“message”: {
“content”: “Peace.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 8,
“message”: {
“content”: “Euphoria.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 9,
“message”: {
“content”: “Dreams.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 10,
“message”: {
“content”: “Awesome! \ud83c\udf89”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 11,
“message”: {
“content”: “Cyclone.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 12,
“message”: {
“content”: “"Hello"”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 13,
“message”: {
“content”: “Apple.”,
“role”: “assistant”
}
},
{
“finish_reason”: “length”,
“index”: 14,
“message”: {
“content”: “Hello. \n\nAs an AI language model, I”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 15,
“message”: {
“content”: “Hello.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 16,
“message”: {
“content”: “Sunflower.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 17,
“message”: {
“content”: “Peace”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 18,
“message”: {
“content”: “Bliss.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 19,
“message”: {
“content”: “Sunshine.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 20,
“message”: {
“content”: “Bubble.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 21,
“message”: {
“content”: “inquisitive”,
“role”: “assistant”
}
},
{
“finish_reason”: “length”,
“index”: 22,
“message”: {
“content”: “Hello! How may I assist you today? Please”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 23,
“message”: {
“content”: “Simplicity.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 24,
“message”: {
“content”: “Hello.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 25,
“message”: {
“content”: “Hello.”,
“role”: “assistant”
}
},
{
“finish_reason”: “length”,
“index”: 26,
“message”: {
“content”: “Hello! How can I assist you? Please provide”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 27,
“message”: {
“content”: “Dream.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 28,
“message”: {
“content”: “Sunshine.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 29,
“message”: {
“content”: “Apple.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 30,
“message”: {
“content”: “Smile.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 31,
“message”: {
“content”: “Coffee.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 32,
“message”: {
“content”: “Peace.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 33,
“message”: {
“content”: “Wonderful.”,
“role”: “assistant”
}
},
{
“finish_reason”: “length”,
“index”: 34,
“message”: {
“content”: “Hello. \n\nI’m sorry, but I cannot”,
“role”: “assistant”
}
},
{
“finish_reason”: “length”,
“index”: 35,
“message”: {
“content”: “Hello! How can I assist you today? Please”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 36,
“message”: {
“content”: “Banana.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 37,
“message”: {
“content”: “Love.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 38,
“message”: {
“content”: “Light.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 39,
“message”: {
“content”: “Stream”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 40,
“message”: {
“content”: “Inquisitive.”,
“role”: “assistant”
}
},
{
“finish_reason”: “length”,
“index”: 41,
“message”: {
“content”: “Hello. \n\nI am a language model AI,”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 42,
“message”: {
“content”: “Dream.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 43,
“message”: {
“content”: “Peace.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 44,
“message”: {
“content”: “Button.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 45,
“message”: {
“content”: “Sunshine”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 46,
“message”: {
“content”: “Innovation.”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 47,
“message”: {
“content”: “Euphoria”,
“role”: “assistant”
}
},
{
“finish_reason”: “stop”,
“index”: 48,
“message”: {
“content”: “Love.”,
“role”: “assistant”
}
},
{
“finish_reason”: “length”,
“index”: 49,
“message”: {
“content”: “Hello. \n\nIf you are looking for a prompt”,
“role”: “assistant”
}
}
],
“created”: 1685131125,
“id”: “chatcmpl-7KXsDrVS7ldv4YTndRvvtbQ5AuCld”,
“model”: “gpt-3.5-turbo-0301”,
“object”: “chat.completion”,
“usage”: {
“completion_tokens”: 197,
“prompt_tokens”: 13,
“total_tokens”: 210
}
}

9

I think it is because of the whole bunch of hallucinations in the second approach. I got this:

[‘Hello. \n\nAs an AI language model, I’, ‘Flower.’, ‘Supercalifragilisticexpialid’, ‘Prompt?’, ‘Sophisticated.’, ‘Pineapple.’, ‘Sparkle.’, ‘Vacation.’, ‘Flower.’, ‘Watermelon.’, ‘Hello! I need more context or instruction to know’, ‘Puzzle.’, ‘Hello.’, ‘Hello.’, ‘Hello. \n\n(Note: This is not a single’, ‘Flower.’, ‘Intrigued.’, ‘Sunset.’, ‘Peace.’, ‘Hello. \n\nI am an AI language model and’, ‘Apple’, ‘Sunshine.’, ‘Supercalifragilisticexpialid’, ‘Robot.’, ‘Ready.’, ‘Serendipity.’, ‘Hope.’, ‘Breathe.’, ‘Bliss.’, ‘Innovation.’, ‘Simplicity.’, ‘Inevitable.’, ‘Sunshine.’, ‘Hello.’, ‘Satisfaction.’, ‘Dream.’, ‘Innovation.’, ‘Snow’, ‘Python.’, ‘Phoenix’, “Hello. \n\nI’m sorry, I cannot write”, ‘Superb.’, ‘Hello.’, ‘Elephant.’, ‘Inspire.’, ‘Palindrome.’, ‘hello’, ‘Cup’, “Hello. \n\nI’m sorry, it seems you”, ‘Hello.’]

And it added a dot to almost every word, which takes the whole token.

I changed the code a bit:

#python
context = "Write the word 'word'."
response = openai.ChatCompletion.create(
      model="gpt-3.5-turbo",
      messages=[
          {
              "role": "system",
              "content": context
          }
      ],
      max_tokens=5,
      n=50
  )

got

[‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word.’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’]

which used only 14 prompt + 51 completion = 65 tokens.
At least we know for sure that it is using tokens for input only once.

By the way, what the hell is ‘Supercalifragilisticexpialidocious,’ and how come it appears in both of our separate outputs?" :smiley:

10

:slight_smile: Looks like a word made famous by Mary Poppins. Now I need to watch the movie:)

Below is the answer from ChatGPT

“Supercalifragilisticexpialidocious” is a word made popular by the 1964 Disney film “Mary Poppins.” It was coined as a song title by the Sherman Brothers, who wrote the music and lyrics for the movie. The word itself is a whimsical creation and is not found in standard dictionaries.

In the context of the film, “Supercalifragilisticexpialidocious” is used to describe something that is extraordinarily good, wonderful, or exceptional. It represents a joyful exclamation and is often associated with a feeling of delight and amazement. While the word is not commonly used in everyday language, it has become iconic as a symbol of something remarkable or extraordinary.

11

Anyway… :smiley:

So to summarize.

#python
context = "Write the word 'word' 50 times, each separated by a space"
response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo",
    messages=[
        {
            "role": "system",
            "content": context
        }
    ],
    max_tokens=250
)

gave me the string of 67(!) words :japanese_ogre:

word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word

And it used a total of 90 tokens, with 23 for the prompt and 67 for the completion.
Which means that ChatGPT seems to have misled me again. It convinced me that one space costs 1 token, but it appears that it costs none :face_with_monocle:

#python
context = "Write the word 'word'."
response = openai.ChatCompletion.create(
      model="gpt-3.5-turbo",
      messages=[
          {
              "role": "system",
              "content": context
          }
      ],
      max_tokens=5,
      n=50
  )

gave me the list

[‘word’, ‘Word.’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’, ‘word’]

And it used a total of 65 tokens, with 14 for the prompt and 51 for the completion.

So, the second approach is actually cheaper, even if it were to give me 50 words (instead of 67).