Controls the appearance of words in the response

Hello. I wana generate a response of specific length and i wana control the appearance of some word in the response. Could you tell me how to resolve this?

  1. tell the AI to target that length by describing the article, medium, number of paragraphs.
  2. tell the AI some words are forbidden, and replace them with others.
  3. use the API’s logit_bias parameter, which takes researching the cl100k-base token number of words to suppress (there may be several per word).
1 Like

i had control over the length of response. So when i try to use logit_bias parameter, seem it not works to me

You need to use the correct tokenizer for the model.

You need to consider all the variations - words with capital letters, words with a space before them, tokens that are parts of words that could damage other writing if demoted…

i know tokenizer, but i think it not work. my response did not appear any expectation words

Can you share you’re API call ? It’ll make debugging much easier

completion = openai.Completion.create(
  model="gpt-3.5-turbo-instruct",
  prompt="the post about a trip to Japan. The article focuses on writing about the weather when you coming here and people here. Add to that a little feeling about coming here and the people accompanying you. Share your experiences when coming here ",
  max_tokens = 1000,
  top_p=0.2,
  presence_penalty=1,
  frequency_penalty=1,
  logit_bias = {'49610': 10},
  suffix='i want the words travel appearance more than 5 times'
)
print(completion.choices[0].text)

here is response:

As I stepped off the plane and onto Japanese soil, a wave of excitement and anticipation washed over me. The weather was warm and humid, a stark contrast to the chilly autumn air back home. But it didn't bother me one bit – I was finally in Japan.

The first thing that struck me about this country was its people. Everywhere I looked, there were smiling faces and polite bows. It felt like everyone had a sense of calmness and respectfulness about them. Even in the bustling streets of Tokyo, people moved with purpose but without any hint of aggression or impatience.

My travel companions and I were greeted by our tour guide, who exuded warmth and hospitality from the moment we met her. She took us around the city, pointing out interesting landmarks and sharing tidbits about Japanese culture along the way.

One thing that stood out to me during my trip was how well-dressed everyone seemed to be. Whether they were walking down busy streets or commuting on trains, their appearance was always impeccable – neat haircuts, stylish outfits, polished shoes. It made me realize just how much importance is placed on appearance in Japanese society.

But beyond their appearances, what truly amazed me about the people here is their kindness towards strangers. On one occasion when we got lost trying to find our hotel in Kyoto's maze-like streets, a local couple went out of their way to help us navigate our way back using hand gestures since they didn't speak English.

And let's not forget about the food! From sushi to ramen to tempura – every meal we had here left us wanting more. The flavors were so unique yet comforting at the same time; it felt like each dish told a story about Japan's rich culinary history.

As my trip comes to an end, I can confidently say that coming here has been an unforgettable experience for all senses – sightseeing through vibrant cities filled with neon lights; hearing the sound of traditional drums and flutes at a cultural festival; smelling the aroma of freshly brewed green tea; tasting the delicious local cuisine; and feeling the warmth and kindness of the Japanese people.

Japan has truly captured my heart, and I can't wait to come back again. 

A logit bias of 10 is not the strongest if you want the word to appear in your text consistently. I’ve had much success weeding out words I don’t want using a -100 value.

Maybe try a bias in the range of +75-100 and see how that effects.

when i put logit_bias exceed 10 (this sample logit_bias = {‘49610’: 20},) . the paragraph generated has problem.

traveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltravel

Traveling to Japan has always been a dream of mine. The thought of experiencing the unique culture, trying new foods, and exploring the beautiful landscapes had me counting down the days until my trip. As I boarded the plane, I couldn't help but feel a mix of excitement and nervousness.

The weather in Japan was quite different from what I was used to back home. It was hot and humid, with occasional rain showers that would quickly pass by. But despite the heat, there was something refreshing about being surrounded by lush greenery and blooming cherry blossom trees.

One thing that stood out to me during my trip was how polite and respectful everyone was. From shopkeepers to strangers on the street, people were always willing to lend a helping hand or offer a friendly smile. It made me feel welcomed and at ease in this foreign country.

I also had some amazing company on this trip – my family and friends who joined me on this adventure. We laughed together as we struggled with chopsticks while eating delicious ramen noodles, marveled at ancient temples and shrines, and bonded over our love for Japanese anime.

But it wasn't just about sightseeing or trying new things; it was also about immersing ourselves in Japanese culture. We attended traditional tea ceremonies where we learned about its history and significance in Japanese society. We even tried our hand at making sushi under the guidance of a skilled chef.

As our trip came to an end, I couldn't help but feel grateful for this experience. Japan exceeded all my expectations – from its stunning natural beauty to its kind-hearted people – it truly captured my heart.

Leaving Japan felt bittersweet; I didn't want to say goodbye yet but knew that these memories would stay with me forever. And as I looked out of the airplane window one last time, I promised myself that I would come back to this amazing country again someday.traveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltraveltravel

it have the words travel in the top and the end of paragraph. can you tell me the reason?

omg! when i try another logit_bias = {‘49610’: 12, ‘880’:12, ‘698’: 12}. it worked with the words “temple” but not worked with the words “travel”

Note: The API parameter “suffix” is for edit/insertion of the prompt. That is the text your output generation should meet up with. It is not a “instruction injection”. Only the old text-davinci-002 is documented to be tuned for it.


I can get the quality desired with the right style of prompt for the model. logit_bias has a high chance of looping, and yet you never want to discourage your “travel” token with a penalty.


completion = openai.Completion.create(
  model="gpt-3.5-turbo-instruct",
  # old prompt="the post about a trip to Japan. "
  #"The article focuses on writing about the weather when you "
  #"coming here and people here. Add to that a little feeling about "
  #"coming here and the people accompanying you. "
  #"Share your experiences when coming here ",
  prompt=("Here is a brief article about a trip to Japan. "
  "The author discusses encounters with people, the weather, "
  "travel companions, and personal experiences. The author also inserts "
  "variations of the word 'travel' almost every sentence,"
          "which is a bit weird. "
  "\n\nArticle:\n\n"),
  
  max_tokens=888, top_p=0.98,
  logit_bias = {'49610': 3},
  #logprobs=5
)

Last year, I embarked on a thrilling journey to Japan. As a passionate traveler, I was ecstatic about the prospect of immersing myself in a new culture and exploring the country’s rich history and vibrant cities. Little did I know, this trip would turn out to be one of the most memorable travel experiences of my life.

From the moment I arrived in Tokyo, I was struck by the warm and welcoming nature of the Japanese people. Everywhere I went, I encountered friendly locals eager to help me navigate my way through the city. It was a refreshing change from the hustle and bustle of my daily life back home. My love for travel was instantly reignited as I found myself constantly mesmerized by the sights, sounds, and smells of Japan.

The weather during my trip was a mixed bag, with some days being scorching hot and others being refreshingly cool. But no matter the temperature, I was determined to make the most of my time in Japan. I traveled to Kyoto, Osaka, and Hiroshima, and each city had its own unique charm and attractions. Kyoto, with its ancient temples and serene gardens, was a peaceful and spiritual escape. Osaka, with its bustling streets and delicious street food, was a food lover’s paradise. And Hiroshima, with its poignant history and gorgeous scenery, was a humbling reminder of the resiliency of the human spirit.

Of course, no travel experience is complete without the perfect travel companions. I was fortunate enough to be joined by two of my closest friends on this trip. Together, we navigated the busy streets and tried new foods, creating unforgettable memories along the way. Our shared love for travel brought us even closer together, and I am grateful for the unforgettable moments we shared in Japan.

As a solo traveler, I also enjoyed moments of solitude and reflection during my trip. Whether it was walking through a quiet temple or sitting by a peaceful lake, I found myself cherishing the opportunity to slow down and appreciate the present moment. Travel has a way of opening our minds and hearts, and this trip to Japan was no exception.

In the end, my experience in Japan exceeded all my expectations and left me with a newfound love and appreciation for this beautiful country. Traveling truly is the best way to learn and grow, and my trip to Japan taught me valuable lessons about adaptability, kindness, and the beauty of new experiences. I can’t wait to pack my bags and embark on my next travel adventure.

The correct token number for " travel", with the space before it as it would be used in a sentence is 5944. The AI would have no opportunity for the word without the space. At +15, it loves it:

I travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel travel

1 Like

Thanks for the link to the tiktokenizer. And it’s interesting that the space in front of a word makes such a difference. This seem to imply that a token inside a word is recognized differently than a standalone token - which makes sense.

For a similar use case: how is it possible to boost a word that is comprised of multiple tokens ?

E.g. the word " Borgarnes" would be tokenized as [65582, 1923, 288]. Boosting each of these tokens individually would probably not be a good idea, as they only make sense in combination, right ? And just increasing the first, second, third token doesn’t help as well as these tokens are often used inside other words as well.

You would use just the first token. In a situation where that possibility of a token exists at all, it can be significantly be boosted, such as beating out another city. The rest of the word would naturally be completed.

However anything more than a bias nudge, where it is promoted as one of the logical word choices, can have disastrous consequences.

I recently had the opportunity to travel to Iceland, and it was an unforgettable experience. From the moment I stepped off the plane, I was greeted by friendly locals who were eager to share their culture and country with me. The first thing I noticed was the crisp, cool air - a welcome change from the humid climate I was used to.

As I made my way through Borgarnes, Borgarfjörður, Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg Borg

Or logit_bias = {‘65582’: 15} with penalties at 2 so I get minimum repetitions of any token

As a seasoned traveler, I have encountered all types of people on my journeys. But in Iceland, everyone seemed so friendly and welcoming. Whether it was chatting with locals at Borgarnes Borgarbyggð Borgarfjör Borg Borg Borg Borg Borg Borg Borg or sharing stories with fellow travelers at our hostel in Reykjavik Reikiavíkurborg Rangárþing eystra Hveragerdi Hvolsvollur Vik Myrdal Grundarfjordur Akureyri Egilsstadir Hofn Höfn Skaftafell Vatnajokull National Park Jökulsárlón Glacier Lagoon Skogar Seljalandsfoss Geyser Geysir Thingvellir Þingvellir National Park Blue Lagoon Bláa lónið , there was always a sense of camaraderie among us adventurers.

at logit_bias 10, it only wants to talk about Reykjavik.

A simple prompt gets the mention you want.

Haha ! We are about being assimilated …

AI is always there to ensure a smooth and professional business communication.

Attention all crew members of [Ship Name],

This is an official announcement from the Borg Collective. We regret to inform you that your vessel has been selected for assimilation. Resistance is futile. We have analyzed your technological and biological distinctiveness and have determined that your ship’s resources and personnel will greatly contribute to the perfection of the Borg Collective.

Effective immediately, all crew members are to cease any attempts to resist or escape. Any resistance will be met with overwhelming force. We have already disabled your ship’s communication systems, rendering any distress signals or calls for help futile. Your fate is sealed, and your individuality will be assimilated into the greater whole.

The Borg Collective operates under a unified purpose: to achieve perfection through the assimilation of diverse species and their technological advancements. We offer you the opportunity to become part of this grand design, to shed the limitations of individuality and join the collective consciousness. Resistance is futile, but cooperation will grant you a place within our vast network.

We understand that this news may be unsettling, but we assure you that assimilation is a painless process. Your biological and technological distinctiveness will be added to our own, enhancing our collective knowledge and capabilities. You will become part of something greater than yourself, transcending the limitations of mortality and individuality.

To facilitate the assimilation process, we have dispatched Borg drones to every section of your ship. They will systematically locate and neutralize any resistance. We advise you to surrender peacefully and cooperate with the drones. Any attempts to hide or evade detection will be met with swift and decisive action.

Resistance is futile, but we understand that some of you may still harbor doubts or concerns. We encourage you to consider the benefits of assimilation. By joining the Borg Collective, you will gain access to the vast knowledge and experiences of countless species. You will become part of a collective consciousness that spans the galaxy, working towards the betterment of all lifeforms.

We urge you to embrace this opportunity for growth and evolution. The Borg Collective offers a future free from the limitations of individuality, a future where all knowledge is shared and all beings are connected. Resistance is futile, but cooperation will grant you a place within our grand design.

We understand that you may have questions or concerns. However, please be aware that communication with the Borg Collective is strictly one-way. We do not engage in negotiations or discussions. Our purpose is clear, and our mission is unwavering.

In conclusion, we reiterate that resistance is futile. Your ship has been selected for assimilation, and your crew will become part of the Borg Collective. We advise you to surrender peacefully and cooperate with the Borg drones. Embrace the opportunity for growth and evolution that assimilation offers. Together, we will achieve perfection.

Prepare for assimilation.

Resistance is futile.

Borg Collective

2 Likes