I’ve been using the o1 Pro Mode consistently, and I’ve noticed a significant decline in its performance today compared to yesterday. Here are the main issues I’m encountering:
Decreased Intelligence: Today, o1 Pro Mode seems much less capable of understanding complex queries. It often provides irrelevant or overly simplistic responses, which is a stark contrast to the more nuanced answers I received yesterday.
Shortened Thinking Time: The processing time has drastically reduced, leading to rushed and incomplete answers. This quick turnaround doesn’t allow for thorough problem-solving or detailed explanations, making it less effective for my needs.
Increased Errors: There have been more frequent inaccuracies and mistakes in the outputs, which undermines the reliability of the tool for important tasks.
These issues are hindering my productivity and overall experience. Has anyone else noticed similar problems with o1 Pro Mode today? If so, do you have any suggestions or solutions to help fix these issues? Any assistance would be greatly appreciated.
Unfortunately I cannot help as I hardly use any o1 model besides mini
Adjust. Adjust, and adjust. That’s the sad truth.
Expect the current flagship model to be constantly changing as OpenAI implements numerous different features for numerous different (good) reasons.
Don’t focus too much on “prompt engineering”, and brute-forcing good results.
It’s unknown how OpenAI manages unexpectedly increased usage, but I do believe other providers have mentioned that they do things like reduce context windows and use quantized models, which can lead to some of the issues you’re describing.
You could set up a benchmark as well. For example if you know that it passed a test yesterday try running the same test today and see how it compares.
I had been using 4o with minmal issues. I upgraded to Pro yesterday and am shocked at how poorly o1 pro is working. It takes a very long time to process. and it does not find current information and argues with you.
Example: I asked for a new laptop recommendation after an Apple rep on a call told me I should be using the new M4 products. o1 pro recomended M3. After I tried to correct it several times int insists that i am the victim of a scam or misinformation. Event that I misunderstood the rep…lol. I went so far as to provide links to Apple product pages, news release on Apple’s site, Product pages on multiple other vendor sites, independent reviews, blogs etc. It still argues insisting I am wrong.
Update: I just tried the same with 4o. It initially gave me the same stale answer by recommending 2023 M3 products. When I provided the same links to Apple product pages, news release on Apple’s site, Product pages on multiple other vendor sites, independent reviews, blogs etc., it corrected its answer and made recommendations for M4 products.
One better - It is not aware of its own existence. It argues that there is no Pro upgrade option for $200/mo and no such thing as o1 Pro.
I have exactly same issues. Canvas, search stopped working. Cant’ analyze images. I think we need to get a discount when chat gpt does not work properly. I have paid $200 and 3 days in the raw can’t use ChatGPT
i have the exact same problems as you with the o1 Pro Mode. it is massively restrictive and makes an incredible amount of extra work.
has OpenAi already done something about it?
I would also like to know what it should do for the price.
@richard52 from what you describe the model is behaving exactly as it should. You didn’t check the cut off date for it’s training or whether it could retrieve links and started asking it about a product that doesn’t exist in the models data using some plain text resembling a link. In response the model exhaustively attempted to inform you that you were wrong, are probably being scammed or are a victim of misinformation. If it were the standard model the finger of blame would be pointed at OpenAI for not preventing you wasting your time with a pop-up or something. But if you’re a pro with a pro subscription using o1 Pro, incessantly bombarding you with basic information whenever you type something resembling a link would be an insult to your intellect and I’m glad they respect this.
It also makes sense that the o1 chat window wouldn’t be aware of it’s existence. The difference between o1 and o1 Pro is the computational resources it is allowed to use. The system decides this not the model. I’d imagine the toggle that lets you switch between the two translates to a flag used by the system when it allocates resources to process the request. If the public facing chat window can access this information it can be exploited. Unsurprisingly it can’t.
It’s not clear why you’d even use o1 Pro for this anyway unless it was accompanied by a five page document detailing nuanced use cases. What benefit would the increased computations have? It may answer the OP though. o1 Pro was meant to be for highly complex nuanced prompts requiring academic level thinking, but if everyone is using it for basic queries it may have led to an increase in computational demand beyond what OpenAI was expecting which led them to reduce its resource utilization.
Yes, I work intensively with different models and thought that for €238 I would get a top model that would help me to develop algorithms. Unfortunately, my experience is sobering and similar to yours.
At the beginning of the session, I got some top results, but then it was like a different world. Without sophisticated prompting, nothing works. But even then, instructions are constantly ignored. Sometimes you get the impression that O1 doesn’t like being pointed out its own mistakes. It basically assumes that the code it creates must be correct.
Another point is that you basically can’t edit projects with O1 Pro because you can’t load files or instructions, how ridiculous. Web search doesn’t work either. You almost get the feeling that it would be a scam if you didn’t know that it was OpenAI. So I prefer to continue using Claude and only use O1 every now and then so that I haven’t paid the €238 for nothing.
So the reality here is that pro is literally designed for professionals with advanced loads. It’s drastically slower than 4o for basic uses because it comes out of the gate with heavy reasoning etc. I have written an automated stock trading bot and an automated crypto trading bot, from scratch, for myself with 0 coding education over the last year. Pro mode is VASTLY superior at delivering me large amounts of code back that are correct the first time whereas 4o will continually need back and forth to get it right. You must use pro mode for what it’s intended for or the functionality will not seem good and frankly Claude is better for the smaller needs that 4o struggles with. I look forward to that changing.
and frankly Claude is better for the smaller needs that 4o struggles with. I look forward to that changing.