OpenAI 1.7.2 - Internal Server Error when using Completions API (Legacy)

API
1

I’m currently experimenting with the OpenAI version 1.7.2 in Python. The Chat Completions API is working perfectly for me, but when I attempt to use the Completions API (Legacy), I encounter the following error:

raise self._make_status_error_from_response(err.response) from None
openai.InternalServerError: Error code: 500 - {'error': {'message': 'internal error', 'type': 'invalid_request_error', 'param': None, 'code': None}}

My code is as follows:

from openai import OpenAI
from decouple import config

API_KEY = config('OPENAI_API_KEY')
model= config('MODEL_ID')
client = OpenAI(api_key=API_KEY)
prompt = "..."
response = client.completions.create(
    model=model,
    prompt=prompt,
)
print(response)

I’m following the documentation at OpenAI Platform. Any insights into why I might be encountering this Internal Server Error with the Completions API (Legacy) would be greatly appreciated.

2

Welcome.

What model are you trying to use? gpt-3.5-turbo-instruct?

3

Hi.

It’s an old fine tuned model. "davinci:ft-..."

4

Davinci went offline 1/24/24… Don’t think the Davinci fine-tunes work anymore…

5

Is there any way to migrate the davinci models to gpt-3.5-turbo, or should I fine tune a new model?

6

With the new OPENAPI the call is for my understanding (as non-Expert)

response = self.client.chat.completions.create(

Fun-Fact: Chatgpt recommended always the old form for API-Call <v1.0

Very strange.

7

That’s not how fine-tuning works, so you’ll need to fine-tune a new model if fine-tuning is a requirement for you.

Old davinci fine-tunes are what now, $0.12 / 1K tokens?

Moving forward there are a few paths you can choose,

  1. Stay with the legacy /completions endpoint and,
    • Use a new base model. One of,
      babbage-002 ($0.0004 / 1K tokens)
      davinci-002 ($0.0020 / 1K tokens)
      gpt-3.5-turbo-instruct ($0.0020 / 1K tokens)
      • Advantages:
        • This has the benefit of requiring the least amount of changes to your code so you can probably be up and running again almost instantly (I think).
        • Probably the cheapest option since there are no fine-tuning fees and the usage fees should be much less ~\frac{1}{60} as much as you’re paying now for usage.
        • It should also be a bit more capable of a model then the base model you were using before.
      • Disadvantage: You lose your fine-tuning.
    • Create a new fine-tuned model using
      babbage-002, (train: $0.0004 / 1K tokens; use: $0.0016 / 1K tokens)
      davinci-002, (train: $0.0060 / 1K tokens; use: $0.0120 / 1K tokens)
      • Advantages:
        • This has the benefit of requiring the least amount of changes to your code so you can probably be up and running again as soon as the fine-tune is ready.
        • Keeps the fine-tune.
        • It should also be a more capable than your prior fine-tune with the improved base.
        • Still much cheaper to use, ~\frac{1}{10} as much as you’re paying now for usage.
      • Disadvantage: You need to re-tune a model which could be expensive and the quality is not guaranteed.
  2. Switch to the new /chat/completions endpoint and switch the model to one of,
    • Use a new base model. One of,
      gpt-3.5-turbo ($0.0020 / 1K tokens)
      gpt-4-1106-preview ($0.03 / 1K tokens)
      gpt-4 ($0.06 / 1K tokens)
      • Advantages:
        • Substantially improved base models improve reasoning and quality of responses.
        • It should also be much more capable of a model then the base model you were using before.
        • Still much cheaper to use, \frac{1}{4} to \frac{1}{2} as much as you’re paying now for usage.
      • Disadvantage:
        • You lose your fine-tuning.
        • You need to re-code all of your API calls which could be a non-trivial, extensive amount of work.
    • Create a new fine-tuned model using
      gpt-3.5-turbo (train: $0.0080 / 1K tokens; use: $0.0060 / 1K tokens)
      • Advantages:
        • Keeps the fine-tune.
        • Still much cheaper to use, ~\frac{1}{20} as much as you’re paying now for usage.
        • It should also be a MUCH more capable than your prior fine-tune with the improved base.
      • Disadvantage:
        • You need to re-code all of your API calls which could be a non-trivial, extensive amount of work.
        • You need to re-tune a model which could be expensive and the quality is not guaranteed.
1 Like