Just noticed this on fine-tuning (May explain a lot)

It looks like fine-tuning uses base davinci and not davinci-003 (You can’t train 003)

And, most important, davinci base model only support 2048 tokens

That means that prompt/completion pairs should be a max of 2048 tokens and not 4096

Yes unfortunately you can only fine-tune base models. I can confirm that I have even tried fine tuning the -00X models (ada through davinci) by passing them as a parameter and it errors out. I wish OpenAI would allow you to build a fine-tune on the more recent -00X models. :frowning:

1 Like