I have been using, what i thought was GPT-4 (by specifying the model=‘gpt-4’). But I wanted to check if i had access to the 32k token limit version. I set the model to ‘gpt-4-32k’ and it said the model did not exist. So I set it back to ‘gpt-4’ and asked (to see) what its token limit was. It answered: ‘The token limit for this model (GPT-3) is 4096 tokens.’
What am I missing?
1 Like
Very few have access to GPT4 32k (different rollout from just GPT4), as of today. Look in Playground or run the model list API endpoint. If you don’t see the 32k model then you don’t have access to it.
You just need to wait, no separate waitlist.
Also don’t ever trust the output of the model on what it is or the limits. It was trained on material before it existed, so why would it know about itself. (Also in general should not be trusted)
1 Like
Thanks for the reply. In the Playground, i do have ‘gpt-4’ and’gpt-4-0314’. I asume the latter one is NOT the 32k token limit version?
I figured the output from the model itself could not be trusted. Does anyone know how to find – well more specifically, verify – the token limit of a model one is using – without just testing it with bigger and bigger prompt to see when it hits the limit?
Thanks! (Had not found that despite searcing.) And I realized just after posting that 0314 was probably a date.
Indeed. gpt-4 is alias for latest, whis is as of today gpt-4-0314 from March 14th 2023
Never ask it about itself, it doesn’t know what it is.
Welcome to the community @andrew21
This topic has been covered multiple times.
Never ask the model its context length nor its name, it doesn’t know.
To know the models available to your org, check playground or use the list models request.
As @novaphil has rightly pointed out, use docs to know the context length.