I have a small Python webapp that uses GPT 3.5 Turbo to generate articles for a blog. Everything is working fine, except that no matter what I do, it will not use the list of keywords I provide while writing the article.
Here is the relevant bit of code.
Class ArticleGenerator():
def __init__(self, keywords):
self.keywords = keywords
self.messages = [
{"role": "user", "content": f'''You must use every word from the following list of words at least
once in the article: {keywords}. Do you understand?'''},
{"role": "assistant", "content": f'''I understand. I will use every word from the list of words
you provided at least once in the article.'''},
]
Iâm using the same method to generate every other part of the article, all of which are contained in self.messages and theyâre working great. This is the only instruction that seems to have no impact at all.
Iâve tried about 10 different ways and locations to include this keyword instruction, but it doesnât work. Any tips would be much appreciated, and I can post more code if the problem isnât clear from above. Thanks!
This is one of the options Iâve tried, but it didnât seem to work either. After a little more research, I think I have to use logit_bias for this, but Iâm not sure how to programmatically tokenize keyword list inputs.
That gave me roughly the same output as the other solutions Iâve tried. The only difference between your example and mine is instead of writing out the keywords [NVidia, AMD, etc], I have them stored in a list that Iâm using in an f string named {keywords}. Would that be making a difference?
If you pass a list object within an f-string, it should appear the same as if you wrote it out in the same spot. You can surround it with your own brackets or quotes to set the list apart.
This is just a case in general of outsmarting the AI and pushing it to the limits of how it creates answers. The language of a prompt isnât going to magically transform the next token it produces in the middle of a sentence into âkeywordâ without a semantic reason for it being there; you might have move up to vital phrases or sentences that need to be in the article at the 1/3 point (for example) to really reshape the generation.
You could take your list of keywords, identify their tokens, then ramp up the logit bias for those tokens to help increase the probability of them being selected.
import openai
openai.api_key = "your key here"
keywords = ['GPT', 'Nvidia', 'Generative AI']
input_messages =[{"role": "system", "content": f'''Generate articles that follow the users instructions exactly.'''},
{"role": "user", "content": f'''Please generate an article using every keyword from the following list of words at least once in the article: {keywords}.
Do not cluster the keywords together. The keywords must be sprinkled through the article in an organic manner, and each one used sparingly.
Ensure the article is at least 500 words long, and no longer than 1000 words.'''},
{"role": "assistant", "content": f'''Here is an article using each keyword in this list {keywords}:
Article: '''}]
print(input_messages)
completion = openai.ChatCompletion.create(
model="gpt-3.5-turbo-16k-0613",
messages=input_messages
)
Edit: Screwed up the spacing when I transferred it into here. Should work now.
I think putting the actual keywords into the system message (instead or as well) would be more likely to get the model to âbehave,â
Alternately (additionally), following the user message with an assistant message saying something like,
Absolutely! Here is an article about XXXX, I will ensure I use the keywords KW1, KW2, and KW3 throughout the article.
By putting the keys and instructions in system, user, and assistant messages it will greatly increase the likelihood of the model choosing to use the keywords.
Lastly, I still think taking the first token in each of the keywords and jacking its logit_bias to 10 or higher will be the most effective way to ensure they get used.
It seemed to do fairly well in my tests of that above. The models understand python quite well, Iâd actually think that it would handle the task better if we leave it with the brackets, because it then has a more formally list to work with.
If generation time or API cost are less a concern than quality, doing what I have above there with a GPT-4 model does even better.
So I tried a few variations of what youâre suggesting here in both GPT-3.5 and 4 and havenât been able to get it to work yet. 3.5 seems to ignore the instruction completely, writing ârefrigeratorâ when âfridgeâ is a keyword, for example. When I tried it in 4, it spun for 10 minutes before timing out.
I believe the issue is that my instructions are overly complex. There are 13 more pairs of user/assistant instructions after what I posted above due to how specific I need the H1s/H2s/H3s and formatting in the paragraphs.
Iâm going to try working with GPT-4 and trimming down my instructions. If that doesnât work, I think looping over the keyword list with tokenizer like @chrstfer is suggesting might do the trick. Thatâs going to flex my newbie Python skills, but Iâll update if I get it working. Thanks for the suggestions everybody.
Quite simply: chat gpt-x models do not follow multi-shot training well (while base GPT-3 rapidly understands). The models are too pretrained to learn. It can sort of work as a mind-warp to make a new personality or beliefs. At most, youâll be giving unwanted biases when it canât diverge from examples but instead recites them.
You should find a way to instruct. Again, making the AI say words it had no reason to interject is a challenge for which I donât have an easy answer other than expanding the on-topic success I showed earlier.
making the AI say words it had no reason to interject is a challenge
Isnât this the point of logit_bias though? Also, it does have a reason to interject them in most cases, i.e., âfridgeâ instead of ârefrigeratorâ.
I donât believe this is something I can instruct. The keywords change every article and they have to be specific for SEO purposes.
With the API, the instructions can change every article and API call, the particulars of operation can be by injected text, and you can have a wall of behavior spelled out to get that list of new words in the new article.
You might even SEO just by telling the AI it is a SEO optimizer and have it extract web metadata and enhance the value of the first paragraphs. SEO is not simply spamming Google with irrelevant words. Search indexers also have intelligence.
I know what SEO is, hence why Iâm working on a program that writes SEO optimized articles. The keywords come from research done prior to article generation, and they must be these specific keywords, not whatever the API feels like spitting out.
the particulars of operation can be by injected text, and you can have a wall of behavior spelled out to get that list of new words in the new article.
Yes, this is what Iâve tested 20 different ways, including your suggestions above, and itâs not working. Whether it be a system message, user message, telling it to rewrite the article and inject keywords, etc., it seems to be ignoring that instruction completely, or at least not knowing what to do with it.
Yeah, thats too much for it. Pare them down. If youâre trying to get it to generate 13 separate articles, then do them all as separate queries. If they need to be related, you can do ask it to summarize between each call. So like, do article 1, then as a separate call to the API, âsummarize this article. Be concise but ensure key details are included. Shorten it significantly.â; then do article 2 and include the summary as "summary of article series so far: " in your system prompt. After you get the second, you send over that summary and the new article for summarization. Wash rinse repeat. This might also save you money because you can do the summarization with a cheaper model.
Also, @_j has a great point with âYou might even SEO just by telling the AI it is a SEO optimizerâŚâ, I bet that would improve the quality of the content a bunch, once you split this monster query into a bunch of smaller ones.
~
You need to separate the logic from the keywords, not just in the code but in your mind. The keywords are interchangable and incidental, they are just a bit of fuel to get an article to come out of the bot the way you want.
You might have more luck by giving it more keywords than you want to appear and letting the bot pick them. Also, if you tell it its an SEO guru it might start swapping words out if it âthinksâ they would fit better. I wouldnt immediately discard that, check with your favorite keyword software/site to see if they actually are. Sometimes its surprisingly good at picking paths I dont see, and while I havent done SEO with it, I really do bet itâs be good at it.
Without knowing your keywords, I canât comment on the best solution, but my advice would be to just do it in steps:
First, tell the model to generate a list of sentences on the topic, one per keyword. This is very easy for GPT in general.
Next, ask the model to use all of those sentences in an article on the topic. Here, I get GPT to use the following keywords: (âbreakfast cerealâ, âbasketballâ, âflashlightâ, âpajamasâ, âfurnitureâ) in an article about graphics cards. Itâs a pretty bad article. You can probably improve easily by providing some system message style direction on style, tone, intended audience, etc.
This works because the first task is something that isnât hard. And, once you have those sentences made, the model has a path through itâs weights and probabilities for tying the intended topic to the keyword every time because it already has the sentences. It can even bend them to flow more naturally on the second generation.
Trying to do the generation all in one go is going to be a task that LLMs are specifically ill-suited to achieve because it forces the generation to keep shifting and jumping to different probability spaces in a way that nothing in itâs training data gives it a roadmap to follow. So, just give it the roadmap ahead of time (that it gives to itself!), and itâll find itâs own way on the second go around.
Why? Maybe it was just what my chatgpt defaulted to and you used gpt3, but if not well⌠cGPT4 is perfectly capable of generating an acceptable, even quite good, article. This kind of thing, having it generate sentences without knowing that theyâre going to be in a single article and then trying to get it to shoehorn them into an article leads to what you got, a weirdly disjointed mishmash of paragraphs with keywords sort of stuffed in there? Like, who thinks of âcomfortâ with a graphics card?
Not trying to be a dick, but if youve got GPT4 this is both unnecessary and detrimental imo. If you want to do this method, generating sentences beforehand, you need a lot of different keywords, then bucket related keywords together and determine a topic beforehand. Iteratively, so it doesnt just list the keywords together either.
Check out this code interpreter completion of my original example. It could 100% be cleaned up to improve it for unguided automation, but i also wanted to give some examples of ways to get the bot to be an effective writer: AI Advancements: GPT-Nvidia Synergy
Compare that with this, which was built off of the link you sent: GPU Insights Unveiled which took longer and was more convoluted.