Is it normal that the same exact query would return wildly different results at different times of the day?

This was mainly noticed today and yesterday and we are just now starting to add in tracking of the queries in more detail so I don’t have specific examples to share yet, but I can get them if this keeps happening but myself and someone working with me are seeing the same exact query give very different answers at different times of the day.

The queries are basically instructions for generating articles and nothing really that complicated just telling it the approximate size of paragraphs, info about formatting and content length etc and then a topic. Also it inserts some placeholder tokens in spots randomly.

The output format is JSON, so it outputs the article and then a few other metadata fields in the JSON.

We were seeing the queries work fine sometimes but other times it would give a very poor output, for example when it was working well it would closely follow the instructions and then give a lengthy article. When it was working poorly though it would create a very short article, like a few hundred words at most, one sentence per section, things like that.

I am in pacific time zone and it seems that the queries that have been behaving poorly have been mostly before 4PM my time, or so, after that it has seemed the quality generally improves for the rest of my evening.

I wasn’t doing much logging in the app I am using that calls the API so I am working on adding that now so I can better track this and report with more solid information but am mainly wondering at this point if anyone else is seeing a similar issue? Thanks for any info.

On new models, OpenAI added the “fingerprint” API output parameter, where they promised to show if there is a model change that would affect determinism. You can log that and see if you are getting a secret high-load lightweight model or something.


@_j very interesting, I didn’t realize that, thanks I will look into logging that