Fine-tuning an OpenAI GPT-3 Model via API for Company Language

[p.bleyleven]

Hello everyone,

I have a project where I want to fine-tune an OpenAI GPT-3 model via API. My aim is to create a model that follows the language of my company. To achieve this, I want to fine-tune it with many texts, but I do not have a specific prompt for the texts.

My question is whether I can simply upload the texts to the API for fine-tuning, and whether it’s better to upload them as a txt file or a JSON file. I would appreciate any advice or recommendations from the community on how to proceed with this task.

Thank you for your help!

[rex.vanhorn]

Hello, @p.bleyleven
I’ve worked on this application myself. As far as I understand, you have to create the jsonl file finetuning, but you leave the prompt column blank, and fill the completion column with your test.
Output from this model will be in the style of your training text.
Good luck!

[curt.kennedy]

If you want the model to repeat facts and knowledge of you company, you need to use embeddings.

If you want the response to have the same general tone and language of your company documents, you need to create prompt-completion pairs by taking chunks of data, using an AI model to neutralize the text and then set your prompt completion for the fine-tune as:

{"prompt": <neutralized text>, "completion": <original text>}

This technique will extract the “personality” of the company, while embeddings will extract the “knowledge” of the company.

Do both if you want both features.

[rex.vanhorn]

Hi, @curt.kennedy
This is awesome!! Do you know if there is any study or paper published on this technique? (I’m happy if a technique just works, but my professors want a little more background research.)

[curt.kennedy]

No paper for tone, other than what was mentioned in this post

Train a GPT model in my tone

Hi, I would approach this by converting as much as possible of your text snippets into a neutral (information only) tone, sort of stripping your language style. This would give you the “common style denominator” format. Train a model to convert text from this “common style denominator” back to your style language. Then when you need to “translate” style from one person to yours, you do it in two steps: You convert their text to neutral style first to keep the information only. You use your…

But try it and see what you get. I have found that these days GPT-4 has the best performance at neutralizing tone. Turbo get’s apologetic, and adds way too much noise instead of following the directions.