There might be some confusion here, especially since this is the ChatGPT category. ChatGPT is the web client. There is an API. ChatGPT translates to the gpt-3.5-turbo or gpt-4 model on the API.
There’s a non-chat model, davinci, the classic “GPT-3”. I’m not sure but I suspect 3.5 to be a heavily fine-tuned model of this. davinci is a bit more raw - it acts more like autocomplete, meaning you don’t ask it to do things.
If you go to the playground, the recommended model would be text-davinci-003. This is an “instruct” model, as in it takes instructions. Similar to ChatGPT, but it’s not fine tuned to refuse certain requests.
Does OpenAI allow fine tuning of the ChatGPT model?
Why this matters is no, you can’t fine-tune the ChatGPT model, but you can tune the underlying davinci and curie models. There’s a tutorial here: https://platform.openai.com/docs/guides/fine-tuning
Basically, you give it a document with expected input and output, and then use that model.
can we expect the fine tuned output to ever match or surpass the quality of say an editor working on the web interface of ChatGPT 4 and trying to improve the output through many back-and-forth rounds of iterations
davinci results are superior to ChatGPT-3.5 in many places, but fine-tuned, it can be expensive and slow. curie can be close, but it depends on how much data you feed it.
They cannot match GPT-4 in terms on reasoning ability, though.
If yes, how could we go about doing that (for ex., using API)?
Try following the link to fine tuning and ask away if you get stuck on it? The API is better in many ways.
You can also consider non-fine tuned chat endpoint. It’s far cheaper than fine tuning and suited to some use cases, like training it to respond to chats in a certain way, or giving specific directions.
If you are planning to create something similar to a FAQ bot, you might consider embeddings instead. Fine-tuning does not retain “memory” it merely guides the autocompletion. Whereas embeddings will be better at answering based on a source.
