Check for gpt-4 access in models list? The data looks like I shouldn't have access, but requests do work?!?

I requested gpt-4 API access and received an email inviting me to use the model a couple weeks ago.

When I check the list of models using my API key like this:
curl https://api.openai.com/v1/models -H "Authorization: Bearer XXX"

The section of the response I get back for gpt-4 looks like I should not have access.
It shows:

    {     
      "id": "gpt-4",
      "object": "model",
      "created": 1678604602,
      "owned_by": "openai", 
      "permission": [
        { 
          "id": "modelperm-cQRoBUgNGW55sPfIdNQh9LA5",
          "object": "model_permission",
          "created": 1682145881,
          "allow_create_engine": false,
          "allow_sampling": false,
          "allow_logprobs": false,
          "allow_search_indices": false,
          "allow_view": false,
          "allow_fine_tuning": false,
          "organization": "*",
          "group": null,
          "is_blocking": false
        }
      ],
      "root": "gpt-4",
      "parent": null
    },
...
    {
      "id": "gpt-4-0314",
      "object": "model",
      "created": 1678604601,
      "owned_by": "openai",
      "permission": [
        { 
          "id": "modelperm-TRTYhl7CtMEjEGEdZaIHNded",
          "object": "model_permission",
          "created": 1682145884,
          "allow_create_engine": false,
          "allow_sampling": false,
          "allow_logprobs": false,
          "allow_search_indices": false,
          "allow_view": false,
          "allow_fine_tuning": false,
          "organization": "*",
          "group": null,
          "is_blocking": false
        }
      ],
      "root": "gpt-4-0314",
      "parent": null
    },

The fact that all the permissions (particularly allow_sampling and allow_view) are false for both gpt-4 and gpt-4-0314 made me think I didn’t actually have access.

But when I make an actual request like this, it works using the same API key:

curl https://api.openai.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer XXX" \
-d '{
"model": "gpt-4",
"messages": [{"role": "user", "content": "why is the sky blue?"}]
}'

Here is the response I get:

{“id”:“chatcmpl-78HBoOWXujIyCRm0Tx6GNuIHBndxN”,“object”:“chat.completion”,“created”:1682207056,“model”:“gpt-4-0314”,“usage”:{“prompt_tokens”:13,“completion_tokens”:223,“total_tokens”:236},“choices”:[{“message”:{“role”:“assistant”,“content”:“The sky appears blue because of the scattering of sunlight by the Earth’s atmosphere. This scattering occurs when the molecules of gases and other small particles in the atmosphere absorb sunlight and then radiate it in all directions. This phenomenon is called Rayleigh scattering.\n\nSunlight is composed of a spectrum of colors, including red, orange, yellow, green, blue, indigo, and violet. When the sunlight passes through the atmosphere, the shorter wavelengths of light, such as violet and blue, are scattered more because they are more easily absorbed by the gas and particle molecules, as compared to the longer wavelengths like red and yellow. However, our eyes are more sensitive to blue light than violet light, so we perceive the sky as blue.\n\nDuring sunrise and sunset, the sky appears reddish because, at those times, the sun is at a lower angle in the sky, and the sunlight has to pass through a thicker layer of the atmosphere. This causes a more significant scattering of shorter wavelengths, and the longer wavelengths like red, orange, and yellow dominate, which we observe as the warm and vivid colors during those times.”},“finish_reason”:“stop”,“index”:0}]}

That does indicate that its coming from gpt-4-0314.

My question is whether there is a way I can check to see if my API key does in fact have access to gpt-4 or other models (without making an actual call to each model to check)?

I expected permissions the data in the /v1/models response to be accurate and use that to determine whether I could use the gpt-4 model, but that does not work. The permissions data looks like I shouldn’t have access, and yet I do?

Or is there user error involved and I’m misinterpreting the permissions data, or there is a different way to check?

1 Like

Testing this again after a while I see that the permissions on an account that has access to gpt-4 has not changed.

Getting the models list using an api key from an account that does not have access, gpt-4 no longer shows up in the list at all.

Is that they way to check?!?

Requesting gpt-4 will get you the latest gpt4 model, which is currently gpt-4-0314. At some point there may be a new version and gpt-4 will point to that, but you could still specify 0314 for awhile

Thanks for the reply, but my goal is to see if an api key has access to gpt-4 in the first place.

When I first looked at this all the keys I had access to returned a similar models list and all the permissions for gpt-4 looked superficially like they should not have access. But some of the keys were from an account that did have gpt-4 beta access and the others did not.

What I see now is that only the api keys that do have access to gpt-4 get data for it in the models list . The values all still say false which would make it seem like they should not be able to use it, but they can.

Now when I get the models list from an account that does not have gpt-4 beta access, it no longer shows any entries for gpt-4 and if I make a request attempt with that model specified, this is the response I get now:

{
    "error": {
        "message": "The model: `gpt-4` does not exist",
        "type": "invalid_request_error",
        "param": null,
        "code": "model_not_found"
    }
}

So yes, I think I’ve answered my own question and the way to check for access is simply to look for the existence of gpt-4/gpt-4-0314 in the models returned for a particular key.

2 Likes

Yep that looks correct :+1:

1 Like