Moving from text-davinci-003 to gpt-3.5-turbo-instruct results in unrecognized suffix error

API
,
1

Hi. I just received an email about gpt-3.5-turbo-instruct. I was eager to try. The following generates completions using text-davinci-0003:

+ curl -sSLH 'Authorization: Bearer ....' -XPOST '-HContent-Type: application/json' -d '{
  "max_tokens": 10,
  "model": "text-davinci-003",
  "prompt": "1 2 3 4",
  "suffix": "7 8 9 10",
  "temperature": 0
}' https://api.openai.com/v1/completions
{
  "warning": "This model version is deprecated. Migrate before January 4, 2024 to avoid disruption of service. Learn more https://platform.openai.com/docs/deprecations",
  "id": "cmpl-80DH3ytuFHgD8IWaI5RlvXiKbJHbN",
  "object": "text_completion",
  "created": 1695061717,
  "model": "text-davinci-003",
  "choices": [
    {
      "text": " 5 6 7 8 9 10\n\n1 2",
      "index": 0,
      "logprobs": null,
      "finish_reason": "length"
    }
  ],
  "usage": {
    "prompt_tokens": 8,
    "completion_tokens": 10,
    "total_tokens": 18
  }
}

However, changing the model fails:

+ curl -sSLH 'Authorization: Bearer ...' -XPOST '-HContent-Type: application/json' -d '{
  "max_tokens": 10,
  "model": "gpt-3.5-turbo-instruct",
  "prompt": "1 2 3 4",
  "suffix": "7 8 9 10",
  "temperature": 0
}' https://api.openai.com/v1/completions
{
  "error": {
    "message": "Unrecognized request argument supplied: suffix",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}

Is this expected? Will it be fixed? Is suffix as an parameter deprecated?

1 Like
2

Welcome to the forum.

The turbo models are the new Chat completion endpoint (ie ChatML - Chat Markup Language…)

/v1/chat/completions gpt-4, gpt-4-0613, gpt-4-32k, gpt-4-32k-0613, gpt-3.5-turbo, gpt-3.5-turbo-0613, gpt-3.5-turbo-16k, gpt-3.5-turbo-16k-0613
/v1/completions (Legacy) text-davinci-003, text-davinci-002, text-davinci-001, text-curie-001, text-babbage-001, text-ada-001, davinci, curie, babbage, ada

You can find more on this page for LLM model endpoint compatibility…

ETA: Seems there’s more info here on Hacker News… I’ve not personally got the email yet.

So… not sure if it’s Chat endpoint or if they’re bringing back legacy endpoint for it? The lower price is appreciated, though!

From HN…

I haven’t checked an email but using the completions api I do get a response using model “gpt-3.5-turbo-instruct”. Interestingly the completions api is deprecated though? Are they bringing it back?

I see it’s briefly mentioned in their docs under the deprecations section.

https://platform.openai.com/docs/deprecations

Note: The recommended replacement, gpt-3.5-turbo-instruct, has not yet launched. Impacted customers will be notified by email when it becomes available.

1 Like
3

It’s possible you are on an Alpha testing list for instruct, but I’ve not heard anything myself yet, when was the email dated?

2 Likes
4

when was the email dated?

“Announcing gpt-3.5-turbo-instruct” - I received it just 1 hour 20 mins ago, at 19:31 CEST time.

1 Like
5

Congrats! Can you try hitting it w/out suffix? Did the email list pricing?

1 Like
6

Can you try hitting it w/out suffix?

Yes, it works without suffix:

$ curl -sSLH "Authorization: Bearer ..." -XPOST '-HContent-Type: application/json' -d '{
  "max_tokens": 10,
  "model": "gpt-3.5-turbo-instruct",
  "prompt": "1 2 3 4",
  "temperature": 0
}' https://api.openai.com/v1/completions
{
  "id": "cmpl-80DgSziiC0xQ9ViA4rYqD8vVVqN9K",
  "object": "text_completion",
  "created": 1695063292,
  "model": "gpt-3.5-turbo-instruct",
  "choices": [
    {
      "text": " 5 6 7 8 9",
      "index": 0,
      "logprobs": null,
      "finish_reason": "length"
    }
  ],
  "usage": {
    "prompt_tokens": 7,
    "completion_tokens": 10,
    "total_tokens": 17
  }
}

Did the email list pricing?

I can just copy it here? I do not think there’s anything sensitive in this:

Hello!

We are excited to announce the release of gpt-3.5-turbo-instruct, our latest model that serves as a replacement for several deprecated models, such as text-davinci-003.

Key Features:
Gpt-3.5-turbo-instruct is an InstructGPT 3.5 class model. It’s trained similarly to previous Instruct models such as the text-davinci series while maintaining the same speed as our turbo models.

Pricing:
We are committed to making cutting-edge technology accessible, so we have priced gpt-3.5-turbo-instruct in line with our other turbo GPT-3.5 models with 4K context.

Thank you for being a part of our journey and for building with OpenAI. Your support enables us to continually advance and bring you the best in AI technology.

Best regards,
The OpenAI team

I am simple man - I do not think I understand what “priced in line” exactly means. I already hit the endpoint, hopefully will not go bankrupt.

@edit

Seems there’s more info here on Hacker News

Och, it is exactly the same text as over there.

1 Like
7

Currently, Turbo is one of the cheapest models…

Model Input Output
4K context $0.0015 / 1K tokens $0.002 / 1K tokens
16K context $0.003 / 1K tokens $0.004 / 1K tokens

The OpenAI pricing page…

I wonder if it’s a new not-deprecated completion endpoint that doesn’t have suffix?

8

Not too sure. It works in the playground but also doesn’t send it in the actual payload :thinking: Maybe they expect us to do it manually? The “inject start text” is added to the payload before actually being sent out. Based on the fact that there was no logprobs added to them makes me believe that yes, we may need to manually do it now.

It could also just have always been the way the playground works :person_shrugging:

Screenshot from 2023-09-18 15-05-45
Screenshot from 2023-09-18 15-05-451223Ă—1045 95.6 KB

Also: Completions is back yussss

1 Like
9

I’m not seeing it here. Is it under Chat or Completion?

ETA: Ah, refreshed and I see it under Completion (Legacy)… The mystery deepens hehe…

10

It’s under Completion

Screenshot from 2023-09-18 15-08-49
Screenshot from 2023-09-18 15-08-491223Ă—145 19.2 KB

1 Like
11

Yeah, refreshed and found it. I do wonder if Completion is coming back, though? Maybe the uproar was loud and they listened?

1 Like
12

Moved to QAPI bugs for visibility

2 Likes
13

Quality at text-davinci-003 style prompting:

Untitled

14

Unfortunately, the completion endpoint doesn’t encode special control tokens such as <|im_sep|> from text, so one can’t completely emulate the chat endpoint (or see if ChatML fine-tune is still there):

json = {"model": "gpt-3.5-turbo-instruct",
        "max_tokens": 1,
        "prompt": "<|im_end|>"}
out = openai.Completion.create(**json)

gives 6 prompt tokens (while 1 token results in = “prompt_tokens”: 1)

{
  "object": "text_completion",
  "model": "gpt-3.5-turbo-instruct",
  "choices": [
    {
      "text": "\n\n",
    }
  ],
  "usage": {
    "prompt_tokens": 6,
    "completion_tokens": 1,
    "total_tokens": 7
  }
}