Output starts often with Linebreaks

General API discussion
1

Hi, i’ve started working with the API but sadly it often gives an output starting with linebreaks… anyone know why this happens? or how i can prevent it?

1 Like
2

Interesting. I have not seen that with my implementation of the completions endpoint. I have found it ignoring or leaving out line breaks.

Do you include line breaks in your prompt/instructions at the start? You may be inadvertently telling it the format you want your response in.

2 Likes
3

Just to clarify, you’re getting responses like this:

[USER]: Hello
[AI]: 
Hi!

instead of:

[USER]: Hello
[AI]: Hi!

?

If so, I’ve had the same issue. I recommend the .strip() string method in Python that removes whitespaces on either end of a given string

1 Like
4

Hey,

I have the same issue and was wondering if somebody has a definite fix for it.

It actually shows this behaviour in the documentation here.

In the example response for a completion the text leads with two line breaks:

  "choices": [
    {
      "text": "\n\nThis is indeed a test",
      "index": 0,
      "logprobs": null,
      "finish_reason": "length"
    }

So is this intended? Why?

Even though Dent’s solution fixes it after the API responded, it still feels like these are 1-2 tokens that could have been put to better use.

5

Didnt found a nice Solution… only ans @paul.armstrong mentioned with replace…

In PHP you can do it like this:

For normal Texts:

preg_replace("/[\r\n]+/", "", $result['choices'][0]['text'])

For lists

for example:

"

  1. Blue
  2. Red
  3. Yellow"

will return an array with each element

$headings = array_filter(explode("\n", $result['choices'][0]['text']), function ($value) {
                    return !empty($value);
                });

if you then want to remove 1. or 1) you can use this regex:

preg_replace(“/^\d+.\s/”, “”, $val);
preg_replace(‘/\s+\d+)\s+/’, ’ ', $val);

But in my opionion this is not the ideal solution as sometimes we might need the linebreaks.

6

in JS you’d just call

choices[0].text.trim()
1 Like
7

I’m pretty sure using “\n” as a stop token would work. I just prefer to have outputs with line-breaks

8

Do you mean logit_bias -100?

I found that solution too, but as you said it would be nice to have appropriate line breaks in the output :slight_smile:

thanks @i-technology and @mionoeldavis for the ideas/workarounds. Still kind of feels a bit off that you waste 2 tokens each time, even though its not the biggest deal.

Maybe it’s about prompt design? Like when phrasing it in a way where it would not naturally use a new paragraph to complete the query?