How would you build a content improver/positive spinning engine on top of GPT-3?

General API discussion
1

We are still very new to GPT-3 so apologies if there is a straight forward answer. We are looking for a way to have GPT-3 output 4-5 different variations of a provided summary text applying different variations. We would push them into a CMS and then have our authors pick the best version including the ability to tweak it.

A prompt is listed below using the davinci engine. A few questions:
— Is it possible to have GPT-3 provide several alternative responses in this use case with one API call or would it require an API call for each version?
— What parameters/models/settings should we focus on to make sure it actually produces the largest breadth of different outputs?
— Is there anything else we are not considering/should consider?

Thank you for helping a GPT-3 noob!

Der 1. FC Nürnberg Verein für Leibesübungen e. V., allgemein bekannt als 1. FC Nürnberg (kurz: 1. FCN oder “der Club”), ist ein traditionsreicher Fußballverein aus Nürnberg, der am 4. Mai 1900 gegründet wurde. Die Vereinsfarben sind rot und weiß, die Traditionsfarben dagegen weinrot und schwarz, in denen er seit der Saison 2007/2008 auch wieder aufläuft. Mit neun Meisterschaften war der “Club” bis 1987 über 60 Jahre lang deutscher Fußball-Rekordmeister, bevor der FC Bayern München ihn ablöste. Gleichzeitig ist er aber auch der bisher einzige Klub, dem das “Kunststück” gelang, ein Jahr nach einer Meisterschaft abzusteigen. Seit Mitte der 1980er-Jahre ist der FCN als “Fahrstuhlmannschaft” bekannt: Es gelingt dem Verein nicht, sich dauerhaft in einer Liga zu etablieren, sondern er steigt wiederholt ab und auf.
“”"
Write a short paragraph about 1.FC Nürnberg.

2 Likes
2

This would be fine with a few-shot example. Once you build up enough examples you could easily fine-tune a model. Input is simply the base material and output would be an improved version. This is essentially neural style transfer, which was figured out before GPT-3.

2 Likes
4

thanks dave, is there a way to formulate a directional prompt and what model should be used.
are you recommending we would actually fine-tune our own model with some of the example of original text and chosen rewrites? are there any good pointers we should consume or concrete examples of this approach?

5

the authors are content writers on our end. so the selection would be made before an end user would consume the content. hope this helps.

6

Yeah so what you’d want to create, ideally, is a training dataset where you have the following format:

Input

200 to 1000 samples of information in various formats, such as blog, news, bullet list, etc
Make sure it ends with a specific phrase or token like “IMPROVED VERSION:”
This unique end token tells the fine-tuned model when to begin the output.

Output

An improved version of each sample based on what it is you’re looking for exactly.
No other framing or tokens are needed, the model will learn to stop at the correct place.

This is the kind of task where you are probably looking for a very specific look and feel to the output. One HUGE advantage of fine-tuning is that you will get very consistent output format and style.

Here are two repos of my own fine-tuning work. Just look for the JSONL files for the examples.