New: Responses API feature - Conversation state API (thread-like replacement)

API
1

Today:

client.conversations.create() - returns a new container id for conversation messages

https://platform.openai.com/docs/api-reference/conversations/create

SDK users will need to update again not just because of client-side parameter validation blocking requests, but because of new endpoint methods.

Usage:

input = (“hello”, as before)
conversation: The conversation that this response belongs to. Items from this conversation are prepended to input_items for this response request. Input items and output items from this response are automatically added to this conversation after this response completes.

truncation - no change, still the maximum that can be billed to you per model.

Documentation

https://platform.openai.com/docs/guides/conversation-state?api-mode=responses

(If you were already doing threads, mandated with assistants, you can continue your pattern of being platform-locked and dependent on a paid service being persistently available for end-user data.)

Note that when OpenAI says, “Response objects are saved for 30 days by default”, that is mistruthful documentation. The actual answer is currently, “invasively forever by default API parameter, with no management API to list them”. Likewise, there is no “list conversations” method available to you with the new endpoint.

image
image797×333 25.1 KB

1 Like
2

Struggling to see the difference between this and just supplying previous_reaponse_idDidnt the previous include all previous responses as well? Seems too…

What is the practical usage for this?

1 Like
3

Feature? You can delete some of the messages and try again.
You can place items without running an AI on them.
You can retrieve them as a unit instead of a separately malformed “last input” and “output” from a response ID.

Previous response ID is an unmanageable state. A conversation ID is mutable and evolving when used.

Documentation of what you will see is low and poor:

  "data": [
    {
      "type": "message",
      "id": "msg_abc",
      "status": "completed",
      "role": "user",
      "content": [
        {"type": "input_text", "text": "Hello!"}
      ]
    }
  ]

Maybe get out some of the “include” items such as reasoning.encrypted_content that were never delivered and blocked when using “store”:“true”, to get yourself back off of “conversations” once you try.

It would be clever if there was some async work done, like extracting a PDF file as soon as it is included in a preliminary user message, but that probably doesn’t happen.

At least for this feature, they didn’t clobber a word already commonly used differently in programming and AI, like “assistant”, “responses”, “prompt”, “thread”, “GPT”, “project”…

1 Like
4

Conversations will become truly valuable when it is possible to configure, in various ways, how much of a conversation is included in the input window.

5

I just have to note:

Absolutely ridiculous about sending "conversation": conversation_id,

  • It does not update the conversation state if using `“store”:“false”
  • The chat still runs, with what was in the messages.
  • You get to figure out why the AI doesn’t remember jack.

That means with store:true, “response id” and all the chat data ran and the AI output is also being stored with every chat turn - despite that you can’t even send “previous_response_id”, even as “null”, when using conversation.

You want only a manageable conversation object as the data stored server side? Doesn’t matter, you get a new response ID also, and if you didn’t log those IDs, your organization is permanently stated up with every call made into the logs. Persistent data ready to leak.

The only solution:

    try:
        with httpx.Client(timeout=600) as client:
            response = client.post(
                "https://api.openai.com/v1/responses",
                headers=HEADERS,
                json=payload,
            )
            response.raise_for_status()
            data: dict = response.json()
            delete_response(data.get("id"))  # immediate ****off, really

A better solution: the experiment with server-side chat history, that grows forever with no budget == done.