Still no way to test this one time without spending $250 for a single api call?
I bumped into this today:
The cost: Pricing
Token Limits: https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo
Basically, 128K tokens, 4096 out and the rest in.
The only thing I changed in my code was the API model name, gpt-4-1106-preview, and the existing code continues to work.
Yeah it looks like all the gpt4 models are only 4k back maximum?
This HURTS if this is the case. The possible 8k tokens i am getting out of gpt4 now is a LIFESAVER I’m some cases.
Someone please tell me I just missed it
1 Like
Someone please tell me I missed too because that would be false advertisement as they did not tell about 4k output and still don’t and hyped everyone by saying 128k context length. openAi would lose prestige on me if something amazing would not pop out in few days.
I think it’s a bit ugly to put the usage of models behind additional paywalls. It makes OpenAI look scammy and less serious in my eyes.
1 Like
Hi there – we’ve just increased rate limits for all users.
You should be able to make at least a couple of large context size requests to GPT-4 Turbo now. As your usage of the API increases, you will move to higher Usage Tiers and you’ll be able to make more large requests to the model.
You can view the new rate limits for GPT-4-Turbo here: OpenAI Platform. You can also view rate limits for your specific account in your account settings here: OpenAI Platform
Really appreciate your patience here as we work through building capacity for the new models.
1 Like
Hello - I am currently in tier 4 but when using the playground for gpt-4-turbo assistant, I receive this error “1 validation error for Request body → content ensure this value has at most 32768 characters (type=value_error.any_str.max_length; limit_value=32768)”
How do I go about activating the 128k token limit?
Feels like their servers are being hammered pretty hard. Giving more people more access is just going to make that worse, so I’d bet it will be a little while before things smooth out and huge calls to the api will work as they should and be available to anyone that wants.
Hey - to try out to the full context window, you can either:
- use the chat completions API with the gpt-4-1106-preview model, or
- use the Assistants API with multiple user messages splitting the text, or
- use the Assistants API with a file which contains the text that you upload and attach that file to the Message
The Assistants API currently has 32768 character limitation per Message in the beta, but we’ll consider expanding this when we GA!
2 Likes
the gpt4 128k context would seem to indicate that we can have in total a whopping 128k in/out in total in a single api request
As I understand it, using such a large capacity model means you can INPUT almost up to 128K tokens, and expect a short answer.
It has occurred to me more often that I needed to input a lot of text, however answers of up to 1024 tokens were more than enough for each response turn.