I’m hacking together an app to generate a comedy news podcast. As such I’m exploring ways to get GPT-4 (and 3.5 turbo 16k) to be actually funny. So far I’ve done this by breaking the show into short sections, each driven by its own prompt. In the prompt are around 8-10 examples of what to write. The output is pretty good, but the prompt is huge. As such the whole podcast script costs around $2 in api fees.
I’m considering instead storing the examples as embeddings and interested to hear thoughts on the pros and cons of that approach. Maybe I’d end up spending the same amount of tokens anyway? And would embeddings be a sensible approach for style reference, rather than the more typical use case of factual retrieval?
It’s generally accepted that humour requires foreknowledge of the punchline/story conclusion, and this is why feed forward neural nets like the GPT models are typically poor at comedy, so any system that lets the model know the punchline (however that is generated) ahead of time will usually produce good results,. I’m not sure what advantage you get from embeddings other than the ability to pick a choose relevant sections from a large amount of existing text.
Now if you were to use the embeddings as the source of the punchlines… that might work, as you could then search for semantically similar content.
I’m currently consulting a company who’s working on a solution you’re looking for. Would be nice to have a chat or a call about the issues you’re having and your goals. The deal would be:
you expose me the workflow your trying to achieve and your issues/goals
That’s a great observation, thanks. Longer-form humour with payoffs and callbacks would indeed be tough without having the model re-read previously-generated content.
Right now though it’s not too bad for the silly news stories I have in mind. GPT-4 wrote this for example:
"Now, let’s move on to tomorrow’s headlines. The Times are leading with "Cat in Disguise Wins Dog Show." The Independent are going with "Pensioner Outruns Lightning in Daring Backyard Dash." The Mail have chosen "Astronaut Discovers New Planet; Forgets Where He Put It." And finally, The Guardian have gone for "Fish Develops Fear of Water; Aquarium Industry in Uproar."
Not hilarious (yet) but very much in the intended style by being fed appropriate examples in the system prompt.
Ok, so lets say you embed all the examples, you will have to have something to search those now embedded examples against, by that I mean, what are you going to retrieve from the examples, and what string of text will you use as the search term? Where is that term coming from?
The reason I ask is that if you don’t ask the right question to the vector database you will not get the right examples out, so there needs to be some way of getting suitable semantic similarity between the embedded data and the thing you are using as a search term. (term here can be more than just one word, it can be an entire page of text if needs be)