One thing to keep in mind is that 4o !== 4o. There’s multiple different minor versions, and some of them are better than others. In my experience, the newer the version, the worse it generally is - but I stopped evaluating after the august '24 version I think.
that is very very likely the case. The API probably doesn’t have access to the specific version that ChatGPT uses. ChatGPT also has a different injected prompt than the API. And depending on the task, OpenAI might be actively working against you (Consuming more tokens than expected for image - Vision - gpt-4o - #4 by _j)