OpenAI Platform the docs mention that the examples for fine-tuning are limited to 4096 tokens is this still true for 16k-context tokens?
File validation would fail on larger example conversation lines.
Pack one with an extra “assistant” role with 4100 tokens of augmentation, and then just nine more placeholders.
I sanitized to 4101 tokens of the AI models list page. Added it as that RAG assistant role before more tokens of system, user, assistant. Whereas a bad json failed, this is in the queue now.
And the fine-tune GUI has no place to chose epochs, so I pay probably 8x more for the little experiment.
1 Like
So lesson: don’t use the GUI to shoot off fine-tune jobs. No suffix. No control over epochs. They could run the thing at your tokens x50 on a whim.
1 Like
Good to know, btw do you know if continuing to finetune a model after the end of a finetuning job will delete the intermediate model? Meaning that if you start a second finetuning job after the end of the second job the intermediate model gets deleted
The first model stays with its original name until you delete it. Just specify that ft:model instead of the original API name model.
1 Like
Nice! But then why am I getting the same model names at least during the second finetuning job?
If you look at the report above, it says base model: gpt-3.5-turbo. I expect that you’d see base model instead as your previous fine tune once underway. Then you receive a new model name when done.
1 Like
Great! Thank you very much for the info 
So anything more than 4k in samples for finetuning doesn’t work?
A bit confused.