Fine-tuning a davinci model to write newsletters

Hi everyone,

I was thinking about how you’d go about fine-tuning GPT3 to write newsletters with different tones and writing styles. I was thinking of building a dataset with about 1000 newsletters containing about 5 different tones to act as completions. I was then going to use a structured approach to developing the prompts i’d use to generate those newsletters myself with reference to the tone and writing style.

I have a few reservations before I attempt this.

Firstly, I can see how fine-tuned models are very effective when it comes to classification or categorisation tasks. However, with completions as long as 200 words it feels like there will be a lot of noise when trying to fine-tune a model to capture the tone and writing style of different types of newsletters. The prompts I would also need to generate would also be relatively long and likely introduce further noise.

Secondly, if I have a dataset of 1000 newsletters at 200 words each, I imagine the token cost for generating the model would be quite expensive. Seems like it would be relatively high risk for something I’m playing around with in my free time.

Does anyone with any experience fine-tuning gpt models for longer form content have any advice? Any feedback on whether my aforementioned concerns are valid would be greatly appreciated.

Apologies if this topic has been covered before, I did look around for a similar topic.


Hi declanobrien96, welcome to the forum!

GPT-3.5-Turbo cannot be fine tuned, nor can GPT-4, only the base models can be, i.e. Davinchi-003 etc. etc.

One of the most effective ways to get a model to speak in a particular way is with a system message to instruct the model how to act, although GPT-4 is better at this, the new gpt-3.5 models also now have better system message following.


In my experience fine-tuning generally isn’t very effective for tasks like that, you’re relying on the base models (davinci etc.) which doesn’t have the capabilities of the newer models, it’s also more expensive.

I’d explore the chat models using those newsletters as templates along with a prompt template describing the desired style (relevant examples can be pulled based on your queries) if you use the embeddings endpoint along with your chat completions calls.

1 Like