How Does OpenAI Manage Context Limit in Relation to the System Message?

API
,
1

Hello everyone, I have a question about how the System Message works in relation to the Context.

I understand that the System Message is sent to provide instructions on how the model should behave.

Let’s suppose the following scenario:

  • SystemMsg
  • UserMsg
  • AssistantMsg
  • UserMsg
  • AssistantMsg
  • UserMsg → At this point, suppose the context reaches its limit.

In this case, I would have to discard older messages to avoid context length errors.

How do different OpenAI products handle this?

Does ChatGPT and the Assistant API remove the previous messages except for the System?
Or do they remove the previous messages, including the System, but the model still retains the system instructions?

1 Like
2

Welcome to the dev forum!

You need to prune on your own with Chat Completion, so you’ll want to take off the oldest user/assistant pairs.