Continuation of conversation over several turns

Hello

I have the Python script shown below for a conversation. The problem is that only one response of person beta is generated. I would like to generate multiple turns until max_tokens is reached. How can this be done?

import openai

prompt = "This is a conversation between person alpha and person beta.\n\npeson alpha:\nHello. How are you doing?\n\nperson beta:\n"

response = openai.Completion.create(
  model="text-davinci-003",
  prompt=prompt,
  temperature=0.9,
  max_tokens=2000,
  top_p=1,
  frequency_penalty=0.5,
  presence_penalty=0.0
)

You have to keep sending the prompt back and add the conversation thus far to it.

There’s a bunch of repositories on GitHub. Here’s some for Python chat with GPT-3…

The code samples should give you an idea of how to go about it.

Hope this helps.

1 Like

Thank you very much.

Sending the prompt back causes much more tokens used compared to creating 2000 tokens with one request.

Let’s say that I can create 30 turns with 2000 tokens. When I have to send the prompt back after each turn, I will need a multiple of 2000 tokens.

Is there no other workaround?

Some people use summarization… That is, they use GPT-3 to summarize the past conversation… It’s not optimal, but it can work at times…

Eventually, I imagine we’ll have a much larger context window available, but this is what we have at the moment.

2 Likes

For example, on textsynth.com one can use GPT-NeoX 20B and it does not stop generating turns for one call to the model (until max tokens are reached). Is this not possible at all with GPT-3?

Oh, you want the API to write both sides of the conversation?

Are you using stop-words? What’s your max_tokens set to?

Yes, I would like to write both sides of the conversation until max tokens is reached.

I’m not using stop words and I set max_tokens to 2000. Please see the code in my initial post above.

prompt = “This is a conversation between person alpha and person beta.\n\npeson alpha:\nHello. How are you doing?\n\nperson beta:\n”

prompt = “This is a long and ongoing conversation between person alpha and person beta.\n\npeson alpha:\nHello. How are you doing?\n\nperson beta:\n”

Could be something as easy as that. Play with the prompt. You might set an explicit stopword like “###” to try to get it to go longer.

Are you trying to create a dataset? Just an experiment?

1 Like

Thank you very much. I’ve now tried with stopword ### and with “a long and ongoing” conversation and other prompt modifications such as “The conversation does not stop” or “The conversation continuous for 10 turns” but it does not work. In the playground, if I try to submit it again, it appears “The model predicted a completion that begins with a stop sequence, resulting in no output. Consider adjusting your prompt or stop sequences.”.

I was now a bit experimenting with the prompt. Sometimes multiple turns are generated but typically no more than 5 turns (although I have set max_tokens to 2000). Does somebody have any idea regarding the prompt? Is there a setting like min_tokens to tell how many tokens the model should produce at least?

Post a short sample of the output so we can see the format you are getting

Are you happy with the format - but just want to be able to stop it cycling 5 turns?

You can either change the prompt or the stop setting. It depends on the output you are looking for

If you post a sample, it will probably be resolved quite quickly

Here is a sample of the output. The prompt was as follows:

This is a conversation between person alpha and person beta. The conversation stops after 50 turns.

person alpha:
Hello. How are you doing?

person beta:

And the generated text (including the prompt is as follows).

This is a conversation between person alpha and person beta. The conversation stops after 50 turns.

person alpha:
Hello. How are you doing?

person beta:
Hi! I’m doing well, thank you. How about you?

person alpha:
I’m doing good as well. So, what have you been up to lately?

person beta:
I’ve been busy with work, but I’ve also been trying to find time for a few hobbies. What about you?

person alpha:
I’ve been busy with work too but I’ve been trying to squeeze in some time to relax. I’ve been playing a bit of video games and reading books. What hobbies have you been trying out?

person beta:
I’ve been taking an online course in photography, and I’ve been trying my hand at a few new recipes. What about you?

person alpha:
That’s really cool! I’ve been learning some new programming languages and playing board games with friends. What have been the highlights of your photography course so far?

In total 224 tokens were generated (including prompt), although I set max_tokens to 2000. The conversation does not continue. Sometimes also just one response or one turn is produced. I would like to produce more turns, as many turns as in max_tokens fit.

max_tokens is designed to stop the AI using tokens, and not to set the length of the output. If you don’t set max_tokens, it defaults to the models maximum

The only way you can increase the output is to change the prompt in some way. Unfortunately that will take some experimentation.

I am sure you have probably tried lots of things already. You could try things like “keep talking until person beta says stop” (which would never happen)

Otherwise, you may have no choice but to use another prompt to continue the conversation

There is no minimum setting you can use.

1 Like

Not perfect, but it’s going for about 1000 tokens… You might try modifying this prompt to fit your needs.

Hope this helps!


This is a long and detailed conversation between person alpha and person beta. The conversation stops after 20 completions which is person alpha and person beta talking. Continue from 1) all the way to 20)

1) person alpha:  Hello. How are you doing?
1) person beta:  Hi! I’m doing well, thank you. How about you?
2) person alpha:  I’m doing good as well. So, what have you been up to lately?
2) person beta: I’ve been busy with work, but I’ve also been trying to find time for a few hobbies. What about you?
3) person alpha: I’ve been busy with work too but I’ve been trying to squeeze in some time to relax. I’ve been playing a bit of video games and reading books. What hobbies have you been trying out?
3) person beta: I’ve been picking up cooking lately. I’ve been experimenting with some recipes and trying to get creative in the kitchen.

4) person alpha: That sounds like fun. What kind of dishes have you been making?
4) person beta: Mostly I’ve been trying out different variations of pasta dishes. I’ve also been trying to make some healthy snacks that don’t require a lot of ingredients.

5) person alpha: That sounds delicious! Have you been enjoying it?
5) person beta: I’ve been having a lot of fun with it. It’s nice to be able to experiment and be creative. It’s also been a great way to relax after a long day.

6) person alpha: It sounds like you’ve been finding ways to stay active and have some fun. What other hobbies have you been trying?
6) person beta: I’ve been trying to get back into painting. I used to love painting when I was younger, so I’ve been picking up some supplies and trying out some new techniques.

7) person alpha: That sounds like a great way to express yourself. Can I see some of your work?
7) person beta: Sure! I’ve been uploading some of my work to my social media accounts. I’d be happy to share it with you.

8) person alpha: That would be great. I would love to see what you’ve been creating.
8) person beta: Of course, I’ll send you the links.

9) person alpha: Thanks so much.
9) person beta: No problem.

10) person alpha: So, what have been your favorite pieces of artwork so far?
10) person beta: I’ve been particularly proud of a few of my abstract pieces. I’ve been experimenting with different color combinations and textures, and I think I’ve really been able to capture the emotion I was trying to convey.

11) person alpha: That sounds amazing. I can’t wait to see them.
11) person beta: I’ll be sure to send you the link as soon as I have a chance.

12) person alpha: Have you also been able to find time to do some other hobbies as well?
12) person beta: Yes, I’ve been trying to find time for some outdoor activities as well. I’ve been hiking and camping, and I’m planning on exploring some new trails soon.

13) person alpha: That sounds like a great way to get out and enjoy nature.
13) person beta: Absolutely! There’s something special about being out in the wilderness and appreciating the beauty of nature.

14) person alpha: What have been some of your favorite places to explore? 
14) person beta: I’ve been to some really breathtaking places. I recently visited Yosemite National Park, and it was just incredible. I’d love to visit some more national parks soon.

15) person alpha: That sounds amazing! Maybe we could plan a trip together sometime.
15) person beta: That would be great! I’d love to explore more of the outdoors with you.

16) person alpha: That would be a lot of fun.
16) person beta: Definitely!

17) person alpha: Anyway, I should probably get going. It was nice talking to you.
17) person beta: It was nice talking to you too. Let’s catch up again soon.

18) person alpha: Sure. Take care!
18) person beta: You too! Bye. 

19) person alpha: Bye.
19) person beta: Bye! 

20) person alpha: Have a great day!
20) person beta: You too! Take care.
1 Like

Thank you very much for your help. I have tried This is a long and detailed conversation between person alpha and person beta. The conversation stops after 20 completions which is person alpha and person beta talking. Continue from 1) all the way to 20).

With this prompt more turns are generated, I appreciate it a lot. But when I’m running it multiple times, it sometimes also stops at 5, sometimes it runs until 10 and so on.