API model set to GPT-4 but seems to respond as GPT-3?

When using the API in python I set the model to GPT-4 as shown below:

 response = openai.ChatCompletion.create(
        model="gpt-4",
        messages=[
            {
                "role": "system",
                "content": "You are a helpful assistant."
            },
            *[
                {"role": "user", "content": msg}
                for msg in conversation_history
            ],
        ],
        max_tokens=2048,
        n=1,
        temperature=0.8,
    )

However when I send a request in Python asking “What gpt model is this?” this is the response:

I am an AI language model based on OpenAI’s GPT-3.

When I ask the same question again on the web version with ChatGPT plus, I get this response:

I am based on the GPT-4 architecture, which is an advanced version of OpenAI’s language models. The GPT-4 model is designed to generate human-like text and is capable of understanding and responding to a wide range of questions and prompts. However, please note that my knowledge is limited to information available up until September 2021.

I confirmed that I have GPT-4 API access after receiving the invite this morning and under ‘usage’ in my account profile, the requests are shown using ‘gpt-4-0314’. But as you can see here the responses are way less detailed compared to the web version. Feels like its not really using GPT-4, anyone else have this problem?

2 Likes

I’m having the same issue. The replies say it’s based on GPT-3, but in the API response it says the model is GPT-4

3 Likes

I was having the same challenge and am seeing similar results.

I have tested both gpt-4 and gpt-4-0314 as the model, and only have the gpt-4-0314 requests show up in my usage.

An interesting observation though, was that my tests using ‘gpt-4’ in my model field, generated a response to my question request / question (although it told me it was gpt-3), yet it did not generate an entry in the Daily usage breakdown, where I can see the gpt-4-0314 appearing.

For me, gpt-4 or gpt-4-0314 doesn’t make a difference, however it’s important to know if we are evaluating the right results from the correct model in order to compare them effectively.

2 Likes

I’ve had a moment of clarity on this. I was looking at it the wrong way.

The appropriate test to see if we have the GPT-4 model, should be to see if it can handle greater than 4,096 tokens.

OpenAI already tell us in the documentation that the model is only trained on data Up to Sep 2021. Therefore, from that dataset, it wouldn’t know what GPT-4 is. While there may have been additional fine-tuning to tell it that it is now GPT-4, that would only be a superficial addition to the model.

The advance, comes not from the training corpus, but from the analysis of that same data in an enhanced way.

The available token count, should surely be the measure for us at this point in time?

It’s my best assessment at this time, however I’m open to see if this is a useful & accurate measuring stick!

1 Like

I’m having the exact same issue. I set the max_tokens to 8096 and it does go through, but I really would like to see if there’s another way to test.

1 Like

it’s a very funny question, the official page says the models was trained till Sep 2021

then i make a cURL request

 curl -v https://api.openai.com/v1/chat/completions -H 'Authorization: MY_API_KEY' -H 'Content-Type: application/json' -d '{ "model": "gpt-4",
"messages": [{"role": "user", "content": "您好"}],
"temperature": 0.7
}' 

then i recived the response below:


well it’s funny ,both the http response header and response body told me it’s gpt-4 replying me.

1 Like

Yeah, even GPT-4 can hallucinate and make up facts, even about itself.

1 Like

Hey! I’m fairly new to all this, but I’m having the same issue in the Playground. It says it’s GPT-3. Also, the replies seem to be way less sophisticated than they are on ChatGPT-4 Plus?

I am not a developer, the only reason I want access to the API is to get the same features as ChatGPT-4 Plus, without the 25 messages per 3 hours limit, since pay as you go seems much more fair to me. But right now, the capabilities of the two models I’m seeing don’t even compare…

1 Like

Were you able to figure out a solution to this issue? I am having a similar issue when using playground. Even though I have gpt-4 as the selected model, the model says its max token limit is 4096.

I have two theories as to the GPT’s responses to questions about itself:

  1. It is never trained on the specifics of itself at this level (“You are GPT-4, now go forth!”), thus, it has to respond confidently and probably inaccurately from the info it it is trained on which only goes to 2 years ago, or whatever. Thus, it could be GPT-4 and just not aware that it is. The API JSON attribute tends to bear this out. This is a hallucination or two types, not being specifically given a piece of information (which I’m not sure it ever is) so it must guess (not to anthropomorphize it), and lack of recent info generally.

  2. I forget what my second theory was but its probably been spun into theory 1.

  3. Oh, here it is, when GPT-4 gets overloaded, us pleebs get responses from lower versions of the thing, which is kinda explained here there any other places.

But it all changes so fast who can know? BTW - I’m just along for the ride just like all of us. I’m just an old dev keeping the brain cells poppin.

-p

1 Like

Typically, the model identity is passed in the system message.

There’s no way a model can inherently know what it is, because the model doesn’t exist until after it is done being trained.

So, the GPT-4 API responding that it is based on GPT-3 is simply a confabulation.

1 Like

You can tell from the lag it is GPT4 OG :confused:

1 Like

Any updates on this?

When I ask gpt-4 API what model it is, it replies, “As an AI developed by OpenAI, I am based on the GPT-3 model.”

When I ask ChatGPT Plus what model it is, it replies, “I’m based on the GPT-4 architecture. How can I assist you today?”

Hi and welcome to the Developer Forum!

The GPT models are trained with data that finishes in September of 2021, at that time GPT-4 did not exist, so the model is not aware that it exists.

ChatGPT has a system message that tells it which model it is, the API lets you define the system message yourself, so you would need to add a system message saying “Yu are GPTR-4 a large language model by OpenAI…”

Please take advantage of the forum search system for other commonly answered question you may have.

hello, im asking the same question to ‘gpt-4-vision-preview’, and it replies that its gpt-3. your reply about 'not knowing bcz its data upto spt 2021 seems not clear here, right? bcz ‘gpt-4-vision-preview’ data is upto Apr 2023, right? and when i ask it ‘can you accept images’? it says ‘no, im only text based model’… why is so?