Is the completion API endpoint stateful in any way?

[sps]

Welcome @shredengineer

@sps

This is to be expected. It is also mentioned in the OpenAI docs .

The API is stochastic by default which means that you might get a slightly different completion every time you call it, even if your prompt stays the same. You can control this behavior with the temperature setting.

I’m not 100% sure on how to fix this but, being very intentional/specific with the prompt and providing examples of how you want it to complete may help.

IMO, fine-tune training data should be focused on getting the desired behavior from the model. Ideally one shouldn’t have to prompt engineer for a fine-tune to get the desired behavior.