Hi, I am attempting to create a chatbot with this API. However, often when the initial conversation prompt is short (sometimes it also happens in longer conversation prompts), the AI automatically completes the next few interactions for me, by this I mean it responds with a few (You:… AI…) pairs. How do I limit the AI to only respond one line and in the format AI:…? Thanks in advance.
Here’s an example of an initial conversation prompt:
The following is a conversation with a japanese friend.
You: こんばんは!
AI: こんばんは! お久しぶりです!
Hi Boris, thank you for your response. I have tried “You:” and it works fine. However, the AI now sometimes appends to my responses instead of responding with “AI:”, is there a solution to this? I have tried setting start_sequence and restart_sequence like so:
start_sequence = "\nAI: "
restart_sequence = "\nYou: "
But to no effect. Any suggestions? Thanks.
Hi and thanks for the reply. What do you mean by “will do whatever reduces the probability of the next token”? As in the model will respond with new tokens that have highest probability of conforming to the patterns found in the prompt?
Sorry I was just under the impression that start_sequence should somehow influence the model to start each response with the defined phrase, I also cannot find much documentation on the exact functions of start_sequence and restart_sequence. Any help? Thanks.
Start and stop sequence are just rules on top of the engine. Stop sequence (you can add up to 4) will monitor produce the equivalent result as if you generated all the requested tokens and then cut out all the tokens following the stop sequence. The restart sequence just means after the completion is finished, append the following text.