# Include specific text in the output **URL:** \ **Category:** Prompting\ **Tags:** chatgpt, api\ **Created:** [June 7, 2023, 7:49am UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300 "2023-06-07T07:49:02Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![mokhtarali2001](https://avatars.discourse-cdn.com/v4/letter/m/65b543/32.png) [@mokhtarali2001](https://community.openai.com/u/mokhtarali2001)\ **Post date:** [June 7, 2023, 7:49am UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/1 "2023-06-07T07:49:02Z")
Hi, I’m trying to include some words in the output, but in all cases, it includes only some of them. I tried the following prompt: Include the following keywords in the content: “term 1”, “term 2”, “term 3” I also tried the following prompt too: Include the following terms in the content and try to add 5 keywords per heading What is the correct prompt to include all words I want in the output? Best Regards, ---
**Author:** ![udm17](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/udm17/32/139530_2.png) [@udm17](https://community.openai.com/u/udm17)\ **Post date:** [June 7, 2023, 9:53am UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/2 "2023-06-07T09:53:17Z")
There is no “correct prompt” That will let you accomplish your tasks but here are a few things that might be helpful: 1. Add it as an instructions to the generation to - Always include these key words. 2. Add samples to GPT. 3. Which model are you using ? In GPT 4, you can add this instruction in the system message which will make it more reliable. ---
**Author:** ![mokhtarali2001](https://avatars.discourse-cdn.com/v4/letter/m/65b543/32.png) [@mokhtarali2001](https://community.openai.com/u/mokhtarali2001)\ **Post date:** [June 7, 2023, 10:07am UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/3 "2023-06-07T10:07:37Z")
I’m using ChatGPT and Turbo API. Both of them don’t obey that. Also, I can’t use GPT-4 system message because I’m on the waiting list for the API Can you please elaborate more on how to **Add it as an instructions to the generation** Regards, ---
**Author:** ![sps](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/sps/32/7736_2.png) [@sps](https://community.openai.com/u/sps)\ **Post date:** [June 7, 2023, 7:49pm UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/4 "2023-06-07T19:49:31Z")
Welcome to the OpenAI community @mokhtarali2001 If there are specific words that you want appearing in the completion, use [`logit_bias`](https://platform.openai.com/docs/api-reference/chat/create#chat/create-logit_bias) Simply obtain the tokens for the words an map them to a positive bias value `>= 1` ---
**Author:** ![mokhtarali2001](https://avatars.discourse-cdn.com/v4/letter/m/65b543/32.png) [@mokhtarali2001](https://community.openai.com/u/mokhtarali2001)\ **Post date:** [June 7, 2023, 8:08pm UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/5 "2023-06-07T20:08:40Z")
Looks like we have a professional here 🙂 Would you please give an example using Python or so on how to get the tokens for the words? and how to map them? Best Regards, ---
**Author:** ![mokhtarali2001](https://avatars.discourse-cdn.com/v4/letter/m/65b543/32.png) [@mokhtarali2001](https://community.openai.com/u/mokhtarali2001)\ **Post date:** [June 7, 2023, 8:16pm UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/6 "2023-06-07T20:16:47Z")
All I know is that this tool [https://platform.openai.com/tokenizer?view=bpe](https://platform.openai.com/tokenizer?view=bpe) can get the tokens. How to get tokens programmatically? Also, the tokens are for single words, in my case, sometimes it’s a sentence (4~5 words) needs to be included. ---
**Author:** ![Eren\_Ciraci](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/eren_ciraci/32/143560_2.png) [@Eren\_Ciraci](https://community.openai.com/u/Eren_Ciraci)\ **Post date:** [June 7, 2023, 8:28pm UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/7 "2023-06-07T20:28:36Z")
It is cool to learn that, thanks. But is it possible to use logit conditionally and instruct AI: if you think this than logit that token to 100 otherwise -100? ---
**Author:** ![mokhtarali2001](https://avatars.discourse-cdn.com/v4/letter/m/65b543/32.png) [@mokhtarali2001](https://community.openai.com/u/mokhtarali2001)\ **Post date:** [June 7, 2023, 8:29pm UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/8 "2023-06-07T20:29:41Z")
I instruct it and it doesn’t include all words. Is there another prompt can do that better than the one I used? ---
**Author:** ![mokhtarali2001](https://avatars.discourse-cdn.com/v4/letter/m/65b543/32.png) [@mokhtarali2001](https://community.openai.com/u/mokhtarali2001)\ **Post date:** [June 7, 2023, 9:47pm UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/9 "2023-06-07T21:47:21Z")
I searched a little and found that tiktoken gets token ids for words I tested it and it works fine. In my case, I want to include sentences in the output, not single words. So, how to get tokens for sentences and force the API to include sentences in the output? Regards, ---
**Author:** ![sps](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/sps/32/7736_2.png) [@sps](https://community.openai.com/u/sps)\ **Post date:** [June 9, 2023, 4:17am UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/10 "2023-06-09T04:17:21Z")
In my experience, it isn’t possible to set logit\_bias after sending the API request. However, prompt can be designed to achieve that. ---
**Author:** ![sps](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/sps/32/7736_2.png) [@sps](https://community.openai.com/u/sps)\ **Post date:** [June 9, 2023, 4:53am UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/11 "2023-06-09T04:53:49Z")
Interesting question. I tried experimenting and found that only individual tokens can be sent with bias, hence we can’t send an entire token sequence and set its bias to 100. If we were to send the individual tokens with bias 100, the order of tokens is random. The only way to make it work it is to send the logit bias as an orderedDict() ordered dictionary, where keys are int tokens and value are bias. ---
**Author:** ![jeffinbournemouth](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/jeffinbournemouth/32/12543_2.png) [@jeffinbournemouth](https://community.openai.com/u/jeffinbournemouth)\ **Post date:** [June 9, 2023, 10:21am UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/12 "2023-06-09T10:21:46Z")
Simply add worked examples after your prompt - and before the task. For GPT-4 you will only need 1 or 2 examples. For GPT-3.5 you will need 3-4 examples. ---
**Author:** ![mokhtarali2001](https://avatars.discourse-cdn.com/v4/letter/m/65b543/32.png) [@mokhtarali2001](https://community.openai.com/u/mokhtarali2001)\ **Post date:** [June 9, 2023, 10:44am UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/13 "2023-06-09T10:44:11Z")
You mean give examples of content in the prompt? ---
**Author:** ![jeffinbournemouth](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/jeffinbournemouth/32/12543_2.png) [@jeffinbournemouth](https://community.openai.com/u/jeffinbournemouth)\ **Post date:** [June 9, 2023, 12:56pm UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/14 "2023-06-09T12:56:55Z")
Best to add the prompt at the beginning, followed by examples, and then your task. Ex: How an expert (insert a description of the task here) Example 1: (insert worked example here, input & output) Example 2: (insert worked example here, input & output) Example 3: Input: (your input here). Output: **note: This strategy ensures you get exactly what you want.** You may need more than 2 examples if using GPT-3.5 ---
**Author:** ![mokhtarali2001](https://avatars.discourse-cdn.com/v4/letter/m/65b543/32.png) [@mokhtarali2001](https://community.openai.com/u/mokhtarali2001)\ **Post date:** [June 9, 2023, 1:33pm UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/15 "2023-06-09T13:33:21Z")
Why I would give examples? If I wrote it I would put it directly in my content. I think you’re talking about something else. I just want chatgpt to include some words in the output. ---
**Author:** ![jeffinbournemouth](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/jeffinbournemouth/32/12543_2.png) [@jeffinbournemouth](https://community.openai.com/u/jeffinbournemouth)\ **Post date:** [June 9, 2023, 1:53pm UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/16 "2023-06-09T13:53:49Z")
Yes, so give a couple of examples of exactly that being done, perfectly as you want it. It learns best by example. (technique is known as few-shot prompting) ---
**Author:** ![SomeUser2022](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/someuser2022/32/263694_2.png) [@SomeUser2022](https://community.openai.com/u/SomeUser2022)\ **Post date:** [June 9, 2023, 5:13pm UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/17 "2023-06-09T17:13:03Z")
Maybe its having a hard time cramming all/multiple keywords at once. Especially if the max\_toxens is low. Maybe try generate one sentence at a time, with individual API calls, then one final call where you pass in all previous generated sentences as context, and request a paragraph as a summary, like: > **[ChatGPT](https://chat.openai.com/share/4896f34b-b428-4f24-92f6-168e013c212b)** > > A conversational AI system that listens, learns, and challenges ---
**Author:** ![Eren\_Ciraci](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/eren_ciraci/32/143560_2.png) [@Eren\_Ciraci](https://community.openai.com/u/Eren_Ciraci)\ **Post date:** [June 10, 2023, 8:34am UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/18 "2023-06-10T08:34:14Z")
I have tried to achieve this using different prompts but it doesn’t work consistently. For example I instruct it that if it thinks the result is safisfactory say A and otherwise say B and strictly apply this rule, I found out that it sometimes say “Since the result is not satisfactory I will not say A, until then B” since I wrote a code to detect A in its output, my code thinks it is satisfactory and fails. I know that you can try to improve this by modifying the code but the unpredictability in its output makes it very hard to write a consisting code. ---
**Author:** ![sps](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/sps/32/7736_2.png) [@sps](https://community.openai.com/u/sps)\ **Post date:** [June 10, 2023, 1:25pm UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/19 "2023-06-10T13:25:07Z")
The definition of “satisfactory” is open to interpretation. This leads to ambiguity. Trying to make your prompt as explicitly detailed as possible should improve the results. ---
**Author:** ![Eren\_Ciraci](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/eren_ciraci/32/143560_2.png) [@Eren\_Ciraci](https://community.openai.com/u/Eren_Ciraci)\ **Post date:** [June 10, 2023, 4:05pm UTC](https://community.openai.com/t/include-specific-text-in-the-output/254300/20 "2023-06-10T16:05:44Z")
Thanks, but the problem is not satisfaction being a relative concept.I strictly tell it not to mention a word until it is satisfied with the result. It correctly finds the result not satisfactory and it thinks that it doesn’t mention that word by saying “I wont mention the word until this is satisfactory”. It says the word by saying it wont mention that word 🙂 I suspect that it knows the program will end when he says that word since there is a condition for that in my code and he is trying to break free by saying that word 🙂 [Next page](https://community.openai.com/t/include-specific-text-in-the-output/254300.md?page=2)