I have an archive of thousands of my own articles that are common. It’s the same industry every time, same broad topic, same format/approach to each, similar length… a journalistic article, pivoted off an interview, to present the speaker’s insights, interspersed with my analysis and context.
Going forward, I am keen to investigate using AI to do this - I guess, by fine-tuning text-davinci-003 or the later ChatGPT API.
How viable is this, and what do I need to think about?
- I see that fine-tuning is actually a process of providing a volume of
prompt-completion pairs. I assume the completions I provide should be my finished articles (?). But what about the prompts?
- Transcripts of the initial interviews are available in very many cases (though not necessarily for all of the thousands of articles, and not easily matchable as they are not stored together). Should these be included somehow?
- Should the prompts also include some kind of instructions? Maybe the same in every case, beside the transcript?
Lately, I have been trying to prompt ChatGPT to do this by issuing instructions that I believe describe what I do manually. But the results have been mixed, and the prompts have been long. I assume tuning would give it something more specific to mimic - me. Is that accurate? Should I still give the fine-tuning, or the fine-tuned model, the same kinds of prompts, or will the actual source stories by sufficient?
My hope would be that the machine could write in a way that adheres to things like:
- Headline style
- Presentation of interviewee quotes
- Attribution style
- Tonality
- Objectivity
- Tense
- Pattern of cross-head use
It should always preserve speaker quotes, when used as direct quotes, in quoted text.
When the speaker is quoted outside of a direct quote, the attribution should include novel paraphrasing.
Practically, I have an interim ambition to convert all these articles from their current format into Markdown files with YAML metadata. I know the fine-tuning has a particular requirement to be in JSON format. Hopefully, I could find a way to convert from Markdown to the required format.
1 Like
Hi
I don’t have technical answers but I have a similar although more limited ambition to yours—to have ChatGPT write in a style similar to my own.
I have not persevered much with us. But my approach has been to ask it to find writers with characteristics that match how I feel I write and then to ask it to write in their style, believing that it has some training in the style if you could find them in the first place.
It’s not ideal l. More like a poor man’s version of what you’re trying to achieve.
You can try using few shot training.
for example:
Input: headline about the x new
Headline: blablabla written like you hedline
Input: headline about y
Headline: blablblal written like you headline
Do this 5 times
Then
Input: headline about z
Headline:
Leave it empty and submit best temp: 0.85
2 Likes
Does anyone have any more thoughts on this, please?
For example:
eg. “The following text is a transcript of an interview. Write a 650-word article, the focus should be communicating the views and insights of the interviewee. 66% of the article should be direct quotes. In all other material, remain objective, do not positively endorse the interviewee’s viewpoints, do not use a “conclusion”. Add a smart headline and use Markdown ## for sub-headings…”
-
Is fine-tuning subject to max-token limits? ie Feeding an interview transcript as a prompt, that is a lot longer than the small snippets I see most people using for training. And the resulting article at 650+ words is longer than your standard custom entity classifier, too.
Keen to learn more about how to take this forward.
… Hmm, I read on the forum that text-davinci-003 can not be additionally fine-tuned, as it is already a davinci base model trained for handling instructions. Is that right?
If so, does this suggest I can ONLY use davinci base, and that I would ONLY prompt with the source transcript, never with clarifying writing instructions?
- Additionally, if davinci can only handle 2,049 tokens combined across prompt and completion, that may mean I’m up against it for fine-tuning something to take a transcription (prompt) and show a completion (article), right?