GPT 3.5 chat completion ignoring stop sequences

API
1

Inspite of me providing the stop sequence, GPT 3.5 is ignoring them. It is continuing with Chat and providing out of context out put. How do I debug this situation

2

Is this is playground or the API ?

In the API, using examples is really helpful personally, I’ve seen it do a better job than a stop sequence to stop the generation, while keeping it in line as well.

If it’s not too much trouble, if you could share your prompt or what you might be trying to do, we could debug that as well.

3

It is not playground. I am using Python, Langchain and the GPT 3.5 model. I am using it as below. I think trying it with tool and agent may be better. I am going to do this now. But the current approach is as below

memory=ConversationBufferMemory()
agent_chain = load_qa_chain(llm,chain_type=“stuff”,memory=memory, prompt=prompt)

response = agent_chain({
“input_documents”:retrieved_documents,
“question”:prompt
},
return_only_outputs=True)

The above approach is giving a lot of inconsistent replies

In the prompt, I mentioned

Answer only based on the context
Do not justify your answer
{context}
{chat_history}
{question}
Answer:

4

The prompt is very bare bones to be honest. Try to add a bit more description of what you are trying to achieve as well as a clear mention of when to stop. And like I mentioned previously, adding a sample or two will greatly benefit the quality of generation. Think of them as the old standard of output that you would want from your system.

5

Finally langchain helped me. I started loving it. I used output formatter from langchain, i think underneath it uses some stop sequences.

1 Like