Newer fingerprints, worst performance

API
1

The first days of gpt-3.5-turbo-0125 there was very good performance, but later i felt it downgrade it, specially it didn’t observe the request to do the step-by-step approach to give the answer.
So i switched back to gpt-3.5-turbo-1106 and started to record the fingerprint: fp_04f9a1eebf. It worked better than 0125, but it happened again now with 1106: it also started to have bad performance and noticed that has a new fingerprint: fp_406be318f3.

What to do in these cases? Why happens that new fingerprint has worst behavior?
How to revert to previous fingerprint?

TIA

PD. My use case: I give LLM the structure of a table and a set of instructions to receive the sql sentence to execute and title to use and some info to make a chart with that data. I ask LLM to go step-by-step to guarantee better performance specially reading the instructions.
I seems that gpt4 almost always works well, but so much slower.

3 Likes
2

The fingerprint is meant to show you when they update the model, by training or algorithm.

It is to tell you when the “deterministic” answers are expected to shift.

There’s no going back unless you can make a widespread case that OpenAI broke the AI, (which hasn’t happened yet on the same model version, despite me telling exactly what day applications were broken as a reply to OpenAI staff).

I was thinking of making an automated topic here that tracks the changes to these model version fingerprints – for the models that provide them.

2 Likes