It looks like fine-tuning uses base davinci and not davinci-003 (You can’t train 003)
And, most important, davinci base model only support 2048 tokens
That means that prompt/completion pairs should be a max of 2048 tokens and not 4096
Yes unfortunately you can only fine-tune base models. I can confirm that I have even tried fine tuning the -00X models (ada through davinci) by passing them as a parameter and it errors out. I wish OpenAI would allow you to build a fine-tune on the more recent -00X models. 
1 Like