How many tokens will I spend if I stop the output with "Stop"?

How many tokens will I spend if I stop the output with “Stop”?

My point is this. I couldn’t prevent the API from writing more than one sentence. And “.” I stop by putting . So, do I spend tokens for one sentence or do I spend tokens for the entire output?

You are in the category for ChatGPT, which is OpenAI’s ChatBot with a web interface. Sounds like this is about the API.

A stop sequence will finish the output of the AI model, and only the language that is sent is the data that is billed. The AI doesn’t keep writing.

A period, and the AI won’t get very far!

Be mindful that the stop sequence is also removed from the AI output. If you receive a finish_reason “stop” you might have to put it back, or the AI may finish for other reasons not needing a period.


Demonstration: I use multiple punctuation as a stop sequence.

Python:

import openai
openai.api_key = key
r = openai.Completion.create(
  model="gpt-3.5-turbo-instruct",
  prompt="Write a cute kitten story.",
  max_tokens=100,
  stop=[".", ",", ";"],
  top_p=0.5
)
print(r)

Output:

“model”: “gpt-3.5-turbo-instruct”,
“choices”: [
{
“text”: “\n\nOnce upon a time”,
“index”: 0,
“logprobs”: null,
“finish_reason”: “stop”
}
],
“usage”: {
“prompt_tokens”: 6,
“completion_tokens”: 5,
“total_tokens”: 11
}

For what little story I got, I was charged for five tokens.

You can see I didn’t even get the comma as stop, so can’t infer if that was supposed to be period, comma, or semicolon (except by reading).


Or I make something it doesn’t stop on the punctuation, and get billed for my max_tokens:

prompt='generate 50 genres. Output json: key="story_genre", value = "\<genre>"',

{
“story_genre”: “Adventure”
“story_genre”: “Romance”
“story_genre”: “Fantasy”
“story_genre”: “Mystery”
“story_genre”: “Thriller”
“story_genre”: “Horror”
“story_genre”: “Sci-Fi”
“story_genre”: “Historical Fiction”
“story_genre”: “Contemporary”
“story_genre”: “Crime”
“story_genre”: “Action”
“story_genre”: “Comedy”
"story_genre

1 Like

Want to promote my prompt demonstration business?