Stop=['\n'] in "gpt-3.5-turbo" leads to 500 error

That may well be, but I have tried repeatedly to confirm this error, including just now, and I cannot get an newline to cause any error.

Example Setup using newline as stop

Example completion (works fine)


I have tried many (at least 10) times with newlines as stops and turbo as the model and have never experienced an error. However, in practice I do not use newlines as stops.



This bug is not limited to newline characters, it happens on the ChatCompletion endpoint whenever the first token generated is a stop token; I ran into it last week but wasn’t sure how to report it.

It can be easily reproduced by setting the logit_bias of any stop token to 100:

  model = "gpt-3.5-turbo",
  messages = [{"role": "user", "content": "hello"}],
  temperature = 0,
  logit_bias = {12340: 100},
  stop = ["!!!"],

And why would anyone earnestly writing API calls submit a logit_bias for a stop?


The bug occurs when a stop token is generated at the start of a response, logit_bias is used here to demonstrate how to consistently reproduce the error to help OpenAI track down what’s causing it on their end.


Thanks for the thread; I set my stop sequence to something else and it fixed the problem. But now
I’m getting similar errors with gpt-4, but they seem to be frequent no matter what I set stop to. I thought it might be due to rate limits, but I don’t always get the RateLimitError. And I don’t have this problem in every application. Pretty baffling.

We had a fairly comprehensive discussion on this logit_bias == 100 in another topic, and our testing showed that the bug was because the API has trouble with high values of 'logit_bias`, stop or not.

Example 1: Hello World, Stop “####”, Logit Bias World, 0

Example 1: Results OK

Example 2: Hello World, Stop “####”, Logit Bias World, 100

Example 2: Results: Pesky Timeout Error Again:

This happens today with large values of logit_bias so I’ll table these tests for now until the models are preforming better.


Note: As this is a lab, I want to see the timeout errors, so I have no internal timeout or retry code active (not production).

Example 3 Results: Hello World, Stop “####”, Logit Bias World, -50 (OK)

Example 4 Results: Hello World, Stop “####”, Logit Bias World, 50

This is consistent with earlier results, where large positive logit_bias values are problematic.

You are correct, evidently I just happened to uncover a second bug with logit_bias while trying to use it to demonstrate the first one. What can I say, I’m good at breaking things.

However, since I do not use a line break as a stop token or logit_bias in the code that keeps triggering this error I’ve done some more testing and it looks like any stop token that simply begins with a line break will cause this.

Here’s a revised example to reproduce the bug, which is still present as of today:

  model = "gpt-3.5-turbo",
  messages = [
    {"role": "user", "content": "Marco"},
  max_tokens = 5,
  temperature = 0,
  stop = ["\nPolo"],
1 Like

I’m good at analysis, coding, building, debugging and testing… so we match :wink:

Here is your example above, tested with your same params:

The Chat Setup

The Results: No Error (Success)



See Also, Appendix:

Same Params, but Using Stop Array:

Results: Success

1 Like

Try using the API directly instead of through a web interface, if I had to guess the input for the stop token is escaping the backslash and treating it as a literal instead of a newline character.

OK !!!

Will test from the command line in the Rails console tomorrow.


1 Like

@ruby_coder i appreciate you looking into this but it seems like the issue is with the python bindings which seem to behave differently

Yeah, I guessed the same. The Ruby API wrapper works very well and I do not use the Python wrapper.


I’m reproducing the same issue with newline and “```” stops with “http post” nushell command (equivalent to curl).
It seems to produce error 500 only if the stop is detected in the response.

it’s now fixed.
see newline char(`\n`) in stop param leads to 500 error in gpt-3.5-turbo · Issue #290 · openai/openai-python · GitHub