Three flavors of System Messages is a bit too much, can the API normalize?

API Feedback
1

Currently:

o1, o3-mini - support developer messages
o1-preview, o1-mini - support no system messages and no developer messages
rest of the models - want system messages

I wonder if there is any way for OpenAI to roll out role: "developer" consistently across all models, to avoid this hoop jumping?

4 Likes
2

They talk about GPT-5 being like the Borg and assimilating all the models … maybe then?

3

I had a double take when I named o1-mini and o1-preview “legacy”, but this is the way the world is rolling now, everything is legacy within 3-6 months.

1 Like
4

I can’t help but think of this.

5

The actual “currently” is:

  • Send either “system” or “developer” role message
  • It is translated to the compatible “authority” message type.
  • (unless o1-preview or o1-mini, not supporting either)

If you are curious and want to exercise the inputs, try interspersing “developer” messages to reasoning models with others, and see if breaking the expected pattern of one main initial message works and has any use for you.

Another curiosity is that max_completion_tokens can be sent as a substitute for max_tokens on previous models – unless you also are using the “prediction” API parameter.

6

I do not think this is accurate… there are so many flavors out there … in extra fun today:

Message (616 copies reported)

DiscourseAi::Completions::Endpoints::OpenAi: status: 400 - body: {
  "error": {
    "message": "Unrecognized request argument supplied: max_completion_tokens",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }`

So turns out that on azure gpt-4o-2024-05-13 does not support max_completion tokens, so now we need special case only to add this for reasoning models.

7

You open up delicious worms. Like the deployment ID, geography, and deployment date, along with Azure API version selected, and what untold translation layers exist.

    "ChatRole": {
      "type": "string",
      "description": "A description of the intended purpose of a message within a chat completions interaction.",
      "enum": [
        "system",
        "assistant",
        "user",
        "function",
        "tool",
        "developer"
      ],
      "x-ms-enum": {
        "name": "ChatRole",
        "modelAsString": true,
        "values": [
          {
            "name": "system",
            "value": "system",
            "description": "The role that instructs or sets the behavior of the assistant."
          },
          {
            "name": "assistant",
            "value": "assistant",
            "description": "The role that provides responses to system-instructed, user-prompted input."
          },
          {
            "name": "user",
            "value": "user",
            "description": "The role that provides input for chat completions."
          },
          {
            "name": "function",
            "value": "function",
            "description": "The role that provides function results for chat completions."
          },
          {
            "name": "tool",
            "value": "tool",
            "description": "The role that represents extension tool activity within a chat completions operation."
          },
          {
            "name": "developer",
            "value": "developer",
            "description": "The role that provides instructions that the model should follow"
          }
        ]
      }
    },
1 Like