o1-Pro is trying to ruin me

ChatGPT
, ,
1

My o1 regular works great, absolutely no issue there at all. Yet at least.

o1 Pro in the first 2 days of purchase wow

o1 Pro now… dumb as muck.

What’s going on? I received a false flag whilst feeding Pro my coding debug logs. And it said I broke terms somehow. That was 1 week ago, and the only consistent thing in my life is that you can rely on o1 Pro being dumb as muck. It’s unusable. I got in touch with OpenAI, but yeah you will wait days for a SINGLE little reply which doesn’t answer your question. Then you reply back, tell them the story again and wait for another few days (praying this time they’ll actually read your latest message fully) and the same thing happens again.

So here’s where I am. I moved over here from Anthropics team plan, been using Claude for the best part of this year, had ChatGPT team plan, as well as Plus. Signed up with poe.com where I pay an annual plan. Been using LLM’s, self-hosted and commercial for past couple of years. I use them exclusively for coding. Not for role-playing, not for any other mental stuff - for coding. Exclusively coding. Okay?.. Good. Therefore I know that any flag that pops up is a false positive. Don’t get me wrong, years ago I would play at breaking ChatGPT’s walls but it got boring because I found it too easy to circumvent. The fun novelty days.

So here I am, just forked out $240 for Pro. Works absolutely magically for 2 days. Then a flag shows. Then flat lines. My Pro suddenly wanted to defy everything I asked it, do the bare minimum, ignore key directives, and be a pia. OpenAI have activated something in Pro which has turned it into the employee from hell. It tries to sabotage. It doesn’t do it’s work. It doesn’t matter how many fresh new chats you use, this thing is out to waste your time in every conceivable way. I know that o1 Pro’s system prompt is either to act dumb, or they have literally switched Pro to 3.5. That is how bad it is. I am sure that OpenAI staff / engineers have a good laugh to themselves setting unleashing a system prompt like this on people, but it’s not funny when you have just paid $240 - it then feels like a nasty fraud.

If you guys thought I was using o1-Pro too much in those first couple of days, fine you could have simply put me on a cool-down… Because I agree, I was using it to comment huge pieces of my code. I made good use of it in those first 2 days. It was surreal to me how much of a jump it was from Claude’s Sonnet 3.5, but nope… You guys have labotomized it, never told me you did so and now made me a voice of discontent which you can just ignore. One little voice, one massive company. I won’t ever be able to fight you guys, but know this - OpenAI, you lot are unethical. And to have you play these kinds of games with not just people’s minds, but with their wallets (and purses), is just nasty. I will not trust you guys to be responsible with AI into the future, when we are only a few years on from chatgpt hitting mainstream and these are the kinds of things you do with customers. :-1: :-1: :-1:

2

I was about to upgrade to the Pro plan, but literally opened the forum summary from the weekly email just before. Now I’d like to hear the opinions of other Pro users, preferably not just “programmers”, but someone who normally uses it for “language” stuff - I usually write legal texts. Thx!

3

Just hold on. OpenAI is still releasing a bunch of new features that may confuse o1. It’s a very rocky road right now but it’s sure to smoothen out. Hopefully once all the releases are out.

4

I subscribed pro. As an applied math researcher, I use it to summarize papers and also simulated real world scenario for me to do decision. For the second part, it is useful but can output many wrong typos and the accuracy is still bad for critical work. If you are looking for low stake work or creative work with some reasoning, it is good. The best use case here is still hack kids’ university homework.

5

Do you think that these issues could be solved through an iterative workflow? As in, having the model (or another model) double-check the work and fix any slight issues?

6

If the question means whether any current industry’s approach could resolve this issue, I think it is impossible.

The current models are outputting answer through probability chain (guided by human expert and training data). There are too many nuances in real world which could make it fail. O1 pro can stumbled on some very easy question like debugging latex code. The “model” they need to improve this is human…

I tried to ask it to make things “accurate” by add more context and nuance. But unfortunately, the improvement is limited. The better practice for me is not improving the model accuracy but explore possibility or asking how the model “ think”. I know asking the real cot process is impossible…

The technology is still impressive and I believe it indeed had some intelligence and great potential in business.

But as end user, the best approach seems to treat it unintelligent but useful. Some business people who promote AI said that the current AI is providing human expert’s level, analysis and service in large scale. If they add words “based on probability”, that would be best description.

I hope openAI put more focus on improving service and products, instead of creating “AGI”—-using complex engineering tricks to beat those benchmark—— which consumes the resources their customers need.

1 Like
7

You read my mind! So glad you said this. I started off thinking 01 Pro was amazing and worth the money. But it’s hallucinating, making up functions i named but never shared, instead of asking me how i use them it just rolls with the fake functions it made up. It thinks it knows and just goes with it. Gives me a wikipedia page full of all this crap when i just want it to help me externalise a function. It’s as bad as GPT40… I may as well go back to using 01-mini… But why am i paying $200USD/month? I can cancel… But what that amazing thing i was using for a few days at the start… I can feel they nuked it… But why don’t they just come clean and admit it. $200/month is a LOT to me.