Is OpenAI punishing people for investing in their platform?

There does seem to be some evidence that the company considers the average user too dumb for it’s best products:

consider this graph from one of their more recent papers:

https://openai.com/index/prover-verifier-games-improve-legibility/

A human will rate a less accurate model as more reliable if it is more legible, because the more accurate model’s output is too difficult to interpret.

This could explain why the LMSYS arena has such weak models at the top of the list. Legibility/interpretability.

https://arena.lmsys.org/ (it’s a stupid gradio SPA, you need to click on leaderboard at the top)

And sama does seem to value the arena:

we try not to get too excited about any one eval, but excited to see GPT-4o mini so close to GPT-4o performance on lmsys at 1/20th the price! https://t.co/5ynjPw29Ls

— Sam Altman (@sama) July 23, 2024

So if OpenAI uses that, and their remote task worker (RTW) consultants’ comprehension as target KPIs, their product strategy from the past year makes a lot of sense: “We have a premier model, but it’s not palatable to vocal users and RTWs - so instead of building stronger, smarter models, we’re gonna build crutches.”

The prompt rewriting thing is a helpful crutch for a new user.

Look at the new JSON thing. It’s a crutch for a crutch. Absolutely unnecessary waste of time and tokens.

Don’t get me started on assistants :confused:

I no longer think it’s bizarre. Just Altman’s (current) strategy.

Developers aren’t important anymore.

1 Like