Fine-tuned model vs last checkpoint

API
,
1

I fine-tuned gpt-3.5-turbo-1106 and now want to fine-tune on the fine-tuned model
When I want to select model I have options to select checkpoints or just the model. When I select last checkpoint as a base model to start fine-tuning it says it isn’t available but selecting just the fine-tuned model as a base model is okay and available. Now I want to know what the difference between the fine tuned model and its last checkpoint is. Is it better to start fine-tuning from fine-tuned model or its last checkpoint or its the same thing?

1 Like
2

OpenAI says:

OpenAI will create one full model checkpoint for you at the end of each training epoch

You can check and report back if there is a checkpoint that duplicates the final state:

  1. List checkpoints - get the count of checkpoints
  2. List fine-tuning job objects - get the n_epochs hyperparameter of the job

If they match, then the last checkpoint is the same as the full model.

If checkpoint count is one less, than there is no checkpoint corresponding to the final model.

In either case, the option you have for continuation is: “you can pass the name of a fine-tuned model into the model parameter when creating a fine-tuning job.” It is unlikely you’d want to start from an earlier checkpoint and continue if that were available, unless you are training on a whole new set of examples that will be activated separately.

2 Likes