Fine tuned model - This is not a chat model and thus not supported in the v1/chat/completions endpoint

API
1

Hi all,

I would really appreciate the help here. I am using the following code in Python, works well with the gpt-4 model, however, once I use my own babbage-2/davinci fine-tuned model, I get this error message:

An error occurred while categorizing: Error code: 404 - {'error': {'message': 'This is not a chat model and thus not supported in the v1/chat/completions endpoint. Did you mean to use v1/completions?', 'type': 'invalid_request_error', 'param': 'model', 'code': None}}

Here is the python code in question:

from openai import OpenAI

def categorize_description(description):
    """
    This function uses OpenAI to categorize the transaction based on its description.
    Args:
        description: The description of the transaction.
    Output:
        category: The category assigned to the transaction.
    """
    if not openai_api_key:
        st.info("Please add your OpenAI API key to continue.")
        st.stop()

    client = OpenAI(api_key=os.environ['OPENAI_API_KEY'])

    prompt = f"'Categorize the following transaction description: {description}'"
    try:
        completion = client.chat.completions.create(
            model="ft:babbage-002:mycustomtrainedmodelhere",
            messages=[
                {"role": "system", "content": "You are a helpful assistant"},
                {"role": "user", "content": prompt}
            ],
            temperature=0.5,
            max_tokens=64,
        )

        category = completion.choices[0].message
        # category = completion['choices'][0]['message']['content']
        return category
    except Exception as e:
        st.error(f"An error occurred while categorizing: {e}")
        return "Uncategorized"

I would appreciate the assistance here.

Thank you.

2

Welcome to the community!

It’s pretty much exactly what the error message says: it’s a completion model, not a chat model. You need to use the raw completion endpoint, not the chat.completion endpoint.

If you look at the docs it’s often marked as “legacy”. This is the OG text prediction mode.

This endpoint also just accepts text, not a conversation object.

1 Like
3

Thanks for taking the time to respond @Diet.

Unfortunately, if you had taken the time to look at the OpenAI docs, you would have realized that what I have done is to the T according to the docs. My reason for reaching out to the community is not because I can’t deduce what the error message is but because maybe, just maybe I missed something.

For your reference, these are the docs - https://platform.openai.com/docs/guides/fine-tuning/create-a-fine-tuned-model

PS: What I have done isn’t marked as “Legacy”.

1 Like
4

Hi!

@Diet’s statement is correct.

You basically need to use your model with the following code for the API call:


completion = client.completions.create(
    model="ft:babbage-002:mycustomtrainedmodelhere",  
    prompt="Your prompt",
    temperature=0.5, 
    max_tokens=64
)

In other words, you need to use the original endpoint for completion models as well as change back from a system/user structure to providing only the prompt.

I hope it is even clearer now.

All the best.

2 Likes
5

This is what I was talking about:

https://platform.openai.com/docs/api-reference/completions/create

image
image1052Ă—653 45.9 KB

it’s just refering to the inference endpoint :slight_smile:

2 Likes