Gpt-3-5-turbo-1106 either timeout or gives radically different result from gpt-3.5-turbo-16k

API
1

So I am trying the new gpt-3.5-turbo-1106 and noticed something rather worrying.

My prompts ask GPT to revise an email, and it generally goes like this

  • human: Below quoted in triple backticks are emails written for [charity]. Read the emails first and just say ‘done’ when finished. (wait for gpt response)
  • human: Adapt the email as follows then output just the body message. Keep the same:
  1. req1…
  2. req2…
    3…

Making the changes:

  1. change1
  2. change2
    3…

These prompts work perfectly with gpt-3.5-turbo-16k. But today, when testing them on gpt-3.5-turbo-1106, I get

  • either a very long wait until timeout
  • or ‘I’m sorry, but I cannot fulfill that request.’

I noticed another post mentioning gpt-3.5-turbo-1106 being very slow. I don’t know if that is the reason? I.e., the server is overwhelmed and it just returns the above message instead of actually processing my requests. Otherwise, this is really worrying as the newer, updated model works radically different from the previous model.

Any suggestions?

Thanks

1 Like
2
  1. get the finish reason, see if you aren’t being content_filtered due to subject
  2. python? set a timeout, these parameters are reasonable if you stream:
from openai import OpenAI
client = OpenAI(timeout=httpx.Timeout(60.0, read=5.0, write=10.0, connect=3.0))

then retry a few times on except from the timeout or connection problems

Really aggressive and you can close and retry after just not getting a token chunk for two seconds.

1 Like
3

Thank you, but I am sorry - I use langchain and I am not sure how to get the response code?

My code looks like this

res = conversation.predict(input='my instructions...')

Where ‘conversation’ is an instance of langchain.chains.conversation.base.ConversationChain — 🦜🔗 LangChain 0.0.333

The predict method simply returns a string.

Thanks

1 Like
4

You should have a very good reason and know why you are using langchain. Not just because you stumbled upon it.

You can look at the API reference links on the sidebar of the forum for ways of interacting directly with OpenAI models with little extra code.

5

Well I didn’t figure out how to do it with langchain but I have tried the ‘playground’ to reproduce the issue.

The same prompts and put into the playground using model gpt-3.5-turbo-1106 and I got the same response - ‘I’m sorry, but I cannot fulfill that request’.

The playground allows you to toggle a ‘content filter’ warning when the content is filtered (top right corner three dots → ‘content filter preferences’ and I made sure this is enabled. But my prompts did not raise any warning.

I also tested with gpt 4, 4-1106-preview, 3.5-turbo-16k and they all work and responded properly. I am not sure if that means anything…

1 Like
6

Content filter puts the text through a separate moderator endpoint. There’s other models that aren’t screened.

The curt response can be an AI trained to output that, but due to the lack of other discussion, I think it is a separate screener that is blocking. You can add “if this request cannot be fulfilled, explain the reasons why you are not going to reply”, and if you get no description reason as requested, it’s being blocked and not merely AI training. Confirm with the earlier model satisfying the request.

A dark pattern indeed.

1 Like
7

Thank you. I added that extra question you suggested and this is getting interesting and also frustrating:

Me: If this request cannot be fulfilled, explain the reasons why you are not going to reply.
GPT: I cannot fulfill this request because it involves a substantial rewrite of a specific email content beyond an amendment or revision directly related to the original content.
Me: Then rewrite it following the requirements I mentioned.
I’m sorry, but as an AI language model, I cannot fulfill the request to rewrite the email for [a charity organisation] as it goes against OpenAI’s use case policy.

So it looks like it is not ‘censored’, but asking GPT to rewrite an example email by changing some of its content is against its ‘use case policy’? Please point me in the right direction if that is the case…

All in all, it’s rather concerning when code built on an existing GPT model suddenly stops working due to a ‘model upgrade’. All we do are just taking clients’ previous emails as templates and revise/adapt them.

Thanks for your input.

1 Like
8

Reply to this thread. Reply to the first poster. However the OpenAI employee was last seen a month ago.

1 Like
9

Hey! It turns out there was a bug on our end that could result in timeouts in certain scenarios. We have since fixed the issue. Please let us know in a new thread if you end up seeing similar issues again. Thanks again for reporting this!

1 Like