Handling of newlines in JSON?

I give this prompt:

{“model”: “text-davinci-003”,“prompt”: “Translate from fr to en => c’est une autre teste\r\nligne 2.”,“max_tokens”: 4038,“temperature”: 1.0}

and get a response missing the “ligne 2” part after the CRLF and containing embedded newlines, not escaped as I would expect in json.
…Dictionary{4}{
“text”:

This is another test.
“index”: 0 …

If I replace the cr & lf with spaces, I get the entire text translated.

  1. How should I send multiline text?
  2. Is the presence of unescaped newlines in the response a bug?

TIA
Patrick

text_to_translate = "C'est une autre teste. \n Ligne 2 \n Ligne 3 \n Ligne 4 "
{“model”: “text-davinci-003”,“prompt”: f“Translate the following lines from French to English:\n {text_to_translate}..”,“max_tokens”: 4038,“temperature”: 1.0}
1 Like

Hi @obeirnepr

You might also consider using a JSON validator for signal JSON lines. You can google for them easy enough.

There are many of them on the net, here are a few examples:

Note: Most of these validators validate JSON data and not JSONL files, so you might need to validate one-line-at-a-time. Or you might get lucky and fine a JSON validator.

:slight_smile:

Thanks Humphree. What makes the difference is the extra space before \n. In the response, the \n newlines are dropped. If I use \r the response has extra blank lines at the start rather than where they are placed in the original. So, in summary, to prepare text for translation I have to remove all CR and ensure there is a space before every LF (and maybe after it too to be sure) before serialising the text in JSON. It’s odd that the response has excess blank lines rather than json \n but if that’s what GPT-3 does I have to work with it.

In case anyone else has the same question, here are my tests and their responses:

  1. \r\n
    “prompt”: “Translate from fr to en => c’est une autre teste\r\nligne 2.”
    Response (note that ‘ligne 2’ is dropped):
    “text”:

This is another test.

  1. No \r, no space before \n
    “prompt”: “Translate from fr to en => c’est une autre teste\nligne 2.”
    Response (note that ‘ligne 2’ is dropped):
    “text”:
    It’s another test.

  2. No \r, space before \n
    “prompt”: “Translate from fr to en => c’est une autre teste \nligne 2.”
    Response: We now have ‘ligne 2’ but a blank line at the start.
    “text”:

This is another test line 2.

  1. Humphree’s prompt:
    “prompt”: "Translate the following lines from French to English:\n C’est une autre teste. \n Ligne 2 \n Ligne 3 \n Ligne 4 "
    Response has a blank line at the start, and all text on one line.
    “text”:

That is another test. Line 2 Line 3 Line 4