Is the completion API endpoint stateful in any way?

I used the completion endpoint (openai.Completion.create(...)) on a fine-tuned davinci model. I ran tests for several hours and got a certain desired behaviour out of a certain prompt consistently during that night. Now, a few days later, the behaviour changed and the model behaves “dumber” than before, even though I’m using the same prompt.

Is the completion endpoint stateful in any way, which could explain how my hour long testing steered the session into the desired behaviour? And a few days later, this state seems to have been reset?

Note that I did not alter the model in the meantime, nor did I change temperature or other parameters of the requests.

Are there external factors beyond my reach which could influence model behaviour over time?

1 Like

It is not stateful.

Every API call goes to a blank slate.

2 Likes

Thank you for verifying. Then I’ll have to think about some other explanation for the behaviour observed…

1 Like

Welcome @shredengineer

IMO, fine-tune training data should be focused on getting the desired behavior from the model. Ideally one shouldn’t have to prompt engineer for a fine-tune to get the desired behavior.

I see. Thank you for quoting the docs on this!