Yeah but many of us are screwed over on getting access. tier 5 required for either model. ChatGPT even limits usage to 30 messages a WEEK which is crazy. While all the elites get to use it and get a head start us normal AI Devs get screwed.
To be fair, this model can be churning a massive amount of tokens that you canât even see. So your input may be 500 tokens, the output is 500, but you get billed for 10,500 output tokens.
This is controllable, but letâs be real. If completely unleashed, people would just try it without implementing any safety measures and then complain about a $10 bill.
You can use a third-party service that hosts this model to try it out if youâd like.
2 Likes
The amount of tokens (10,500) is fine. Thatâs what most agents do for example when solving a complex prompt and this new model makes me wonder how much better agents can be with a model like this. The elites get access and get to be experts on it before us normal people get access to the models. We are trying to create products to compete with the elites products which is a 100% lost battle when they get such a time advantage. Iâll try to find a third-party service that hosts this model to be able to keep up.
2 Likes
Agreed.
Itâs unfair to give this to people who paid more. The depth of your pocket is synonymous to âtrustâ according to OpenAI. They may be right, who knows.
I wouldnât worry about this model providing a competitive edge unless your product is a GPT wrapper.
I think here is a fair analogy:
Who would you trust to provide better results?
- Someone who has a fair understanding of the topic, and has access to all the tools and communications necessary to form a response
OR
- Someone very reflective but has to think of a solution on the spot with no help , no tooling, and doesnât tell you how they reached the conclusion?
Obviously the best solution is a combination of both, but as of now, this is not what the model is.
It is really good!
Logical thinking is top notch. It seems to be more than just a chain of thought upgrade. It seems to understand whatâs being asked of it more fundamentally.
2 Likes
Right!! I love it so much!! Holy cow its good! Super happy with this new Q* aka strawberry aka o1 model!
1 Like
Every Model has its place. All the way from Open Source Models like llama 3.1 to Claude and Open AI models. The main question to answer is what types of agents or prompts this new model excels at better than the others. Iâll not know the answer to that question until I can spend some time with it and 30 messages a week will not cut it. I can burn through those in an hour lol. Plus doing it through chatgpt is not true testing. Hopefully Iâll find a third-party service that hosts this model in the next day or so.
1 Like
Absolutely.
If âreflectionâ is the next trick of LLMs then we will see a lot of these models start to pop up and see some very clever agentic frameworks that use it.
1 Like
Iâm posting a whole series of o1 vs Claude comparisons to X⌠o1 may be good at complex math but it still canât count 
1 Like
Thereâs no rush to change your code. We will continue to support max_tokens forever on existing models. However, the change in nature of output tokens with o1 can break certain clients in production. To avoid this, we are requiring the new parameter. More here: https://platform.openai.com/docs/guides/reasoning/controlling-costs
4 Likes
Will the new parameter also become available in the Playground?
Does it allow use of "Structured Output"s?
1 Like
How does making customers change the word âmax_tokensâ to âmax_completion_tokensâ prevent breaking them? They already had to change âmodel: gpt-4oâ to âmodel: o1â. Youâre just saying you have to change two parameters to use the new model and not one.
My issue is, the me as an SDK developer (the Microsoft Teams AI Library), I now have to ship a patch to my SDK before customers can use this new model. Iâm assuming most SDK developers are going to do what I did and just do a search and replace to change âmax_tokensâ to âmax_completion_tokensâ. Iâm sorry, I just donât understand what you think this change is achieving.
I get that you want to make sure that customers know theyâre going to be charged more for hidden output tokens but changing this parameter name isnât going to achieve that. Itâs just creating a pain point for developers. And I say that from spending the last decade designing SDKs.
The o1 models introduce reasoning tokens . The models use these reasoning tokens to âthinkâ, breaking down their understanding of the prompt and considering multiple approaches to generating a response. After generating reasoning tokens, the model produces an answer as visible completion tokens, and discards the reasoning tokens from its context.
@stevenic found this on https://platform.openai.com/docs/guides/reasoning, I guess theyâre trying to distinguish the different between reasoning and completion tokens
2 Likes
i hope Hallucination and biases it not in this model
One has to request access on Azure
So I guess I wonât hold my breath
Thanks for the link anyway
1 Like
NO - it is on Oct 2023 cutoff - but hey is absurdly powerful. Just tested it.
Sorry, i donât understand
I tried to access the playground on Azure, and it asked me to submit a request with the usual blah blah saying pretty much that they have no idea when access will be given!
After 3 attempts, o1-preview was able to solve the question. Same prompt. GPT-4o, Sonnet3.5, and other AIs were not able to solve it.
This was surprising.
The reason given is:
Because some applications might rely on
max_tokensmatching the number of tokens received from the APIâŚ
Iâm struggling to think of any use cases where an application would match the number of tokens received from the API to max_tokens. Does anyone have any ideas about this?