It has o1-preview as a “snapshot version” of o1. Quite similar to how o1-preview is listed in the pricing page under the category “o1”. However, the features that had just been listed for o1 are quite different for o1-preview, it having its own comparison model page also.
Then we get to the rate limits for o1. Organizations are actually being given 10% of that rate shown.
Then, the fields of “supported features” are especially odd. Do we really need to know that 20 different AI models don’t support “inpainting”? I picked DALL-E 2 to see if “edits” endpoint corresponds to inpainting, and no, the only model that could refer to inpainting (or outpainting) has it unchecked.
Then we come to more “features” problems. GPT-4o is all piled together. They don’t have the same features though. That “structured outputs” showing below doesn’t apply to the 2024-05-13 model listed immediately after (though it does take json_object as response format just like gpt-4-turbo). Then try to send a prediction parameter to it also…
It seems this and the pricing all comes from a database. With the original pricing page now destroyed, we can’t see if gpt-4o-audio-preview-2024-12-17 is cheaper than 2024-10-13 - and can’t see the price of $200.00 / 1M tokens of audio at all.
This must be reference quality before you take down the reference.
The phrase “so that performance and behavior remain consistent” is a blatant mistruth. We’re approaching two years of OpenAI breaking API applications overnight that use the versioned models with stealth changes to the models.
I see that at the very bottom, the orange that is “lit up” is a selected model, and it changes the previous information. That you can click in a small specific spot and switch a sub-model (in some cases to a different one that has its own page) is quite non-discoverable, and counter intuitive that it changes the information that came before.
It seems some effort was made to add some information about the audio input cost. However, it is wrong in its audio comparison, and without the version of the model being shown, there being two different input pricings currently:
The previous pricing page had a cost estimate per minute, now missing. This could better be described for developers as a tokenization estimate per minute (and per repeated context input run containing any and all audio).
Hi, thanks for taking the time to write all of this feedback, super helpful!
On snapshots: you are totally right that it is confusing to have things like o1-preview displayed in the same way as snapshots. We tried to make it visually different, to show that it’s not a snapshot but a “related model”, but it’s not obvious and we’ll take it into account to redesign this section.
When you click on o1-preview, the data on the page (rate limits, pricing, features etc) changes accordingly.
For features: the “supported features” are actually linked to a family of models, so things like inpainting don’t show up for other models than DallE in the individual model pages, but we do need to apply this same logic to the compare page as well, thanks for flagging!
And if you click on a snapshot on the model page, you will see that the features supported and pricing adjust to the current snapshot. So if you click on the gpt-4o-2024-05-13 snapshot, you will see that Structured Outputs are not supported.
