Is it possible to fine tune ChatGPT 3.5 for our specific usecase? (we've read mixed answers)

We develop medical content at scale on various niche health topics. So far what we’ve found is that neither ChatGPT 3.5 nor ChatGPT4 manages to produce content of the quality that our writers (mostly MDs) provide. More specifically, its not the data but the style, format, flow of logic, balance of urgency to empathy, etc. that seems to need a lot of work.

From what we understand, one way to improve the quality of output is by fine tuning the model. So here is our question…

  • Does OpenAI allow fine tuning of the ChatGPT model?

  • If you are allowed fine tuning of ChatGPT but only on the previous versions like ChatGPT 3 or ChatGPT 3.5, can we expect the fine tuned output to ever match or surpass the quality of say an editor working on the web interface of ChatGPT 4 and trying to improve the output through many back-and-forth rounds of iterations?

  • If yes, how could we go about doing that (for ex., using API)?

Thank you in advance.

There might be some confusion here, especially since this is the ChatGPT category. ChatGPT is the web client. There is an API. ChatGPT translates to the gpt-3.5-turbo or gpt-4 model on the API.

There’s a non-chat model, davinci, the classic “GPT-3”. I’m not sure but I suspect 3.5 to be a heavily fine-tuned model of this. davinci is a bit more raw - it acts more like autocomplete, meaning you don’t ask it to do things.

If you go to the playground, the recommended model would be text-davinci-003. This is an “instruct” model, as in it takes instructions. Similar to ChatGPT, but it’s not fine tuned to refuse certain requests.

Does OpenAI allow fine tuning of the ChatGPT model?

Why this matters is no, you can’t fine-tune the ChatGPT model, but you can tune the underlying davinci and curie models. There’s a tutorial here: https://platform.openai.com/docs/guides/fine-tuning

Basically, you give it a document with expected input and output, and then use that model.

can we expect the fine tuned output to ever match or surpass the quality of say an editor working on the web interface of ChatGPT 4 and trying to improve the output through many back-and-forth rounds of iterations

davinci results are superior to ChatGPT-3.5 in many places, but fine-tuned, it can be expensive and slow. curie can be close, but it depends on how much data you feed it.

They cannot match GPT-4 in terms on reasoning ability, though.

If yes, how could we go about doing that (for ex., using API)?

Try following the link to fine tuning and ask away if you get stuck on it? The API is better in many ways.

You can also consider non-fine tuned chat endpoint. It’s far cheaper than fine tuning and suited to some use cases, like training it to respond to chats in a certain way, or giving specific directions.

If you are planning to create something similar to a FAQ bot, you might consider embeddings instead. Fine-tuning does not retain “memory” it merely guides the autocompletion. Whereas embeddings will be better at answering based on a source.

1 Like