A bug that causes GPT-4 to be unresponsive for long

API
1

I encountered an issue while using GPT-4o for translation. When translating specific text, it becomes unresponsive for more than 3 minutes and outputs truncated JSON content.

curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-4o-2024-08-06",
    "temperature": 0,
    "messages": [
      {
        "role": "user",
        "content": "Your task is to translate the provided source text to Spanish.\nYou must not translate Arabic numerals or Roman numerals.\n\n\n\nPlease translate the following text:\n13.10 Other Faults.                      .192                      13.10 Other Faults.                      .192\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t13.10 Other Faults.                      .192                      13.10 Other Faults.                      .192"
      }
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "result",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "translation": {
              "type": "string"
            }
          },
          "required": [
            "translation"
          ],
          "additionalProperties": false
        }
      }
    }
  }'
1 Like
2

@purefda I just tried your request and worked fine and almost instantaneous.

I also replicated it with Python/Pydantic and that worked fine too:

s = "Your task is to translate the provided source text to Spanish.\nYou must not translate Arabic numerals or Roman numerals.\n\n\n\nPlease translate the following text:\n13.10 Other Faults.                      .192                      13.10 Other Faults.                      .192\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t13.10 Other Faults.                      .192                      13.10 Other Faults.                      .192"

class Translation(BaseModel):
    translation: str

response = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    messages=[
        {
            "role": "user",
            "content": s
        }
    ],
    response_format=Translation
)

I get the following response:

pprint.pprint(json.loads(response.choices[0].message.content), indent=4)

{   'translation': '13.10 Otros Fallos.                      '
                   '.192                      13.10 Otros '
                   'Fallos.                      .192\n'
                   '\n'
                   '\n'
                   '13.10 Otros Fallos.                      '
                   '.192                      13.10 Otros '
                   'Fallos.                      .192'}
3

I copy and pasted on ChaptGPT 4o Mini and it was instant results. I think maybe the local sever used is causing errors (temp) or try using another browser without blockers and script extensions (pause) them for the moment and try it. I hope this helps my friend.

4

It seems fixed now. But after adding a system prompt, the bug has been reproduced:

curl https://api.openai.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-4o-2024-08-06",
    "temperature": 0,
    "top_p": 0.8,
    "messages": [
      {
        "role": "system",
        "content": "You are a medical document translation expert."
      },
      {
        "role": "user",
        "content": "Your task is to translate the provided source text to Spanish.\nUse professional terms and adhere to the Medical terminology standard, particularly those related to clinical trials.\nYou must not translate Arabic numerals or Roman numerals.\n\n\n\nPlease translate the following text:\n13.10 Other Faults.                      .192                      13.10 Other Faults.                      .192\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t13.10 Other Faults.                      .192                      13.10 Other Faults.                      .192"
      }
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "result",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "translation": {
              "type": "string"
            }
          },
          "required": [
            "translation"
          ],
          "additionalProperties": false
        }
      }
    }
  }'
5

The bug is triggered only when using GPT-4o-2024-08-06 and specifying response_format=json_schema.