Responses API: Question about managing conversation state with previous_response_id

API
1

I’ve been tinkering and want to make sure I’m tracking correctly. It appears that managing conversation state on the server side by passing the
previous_response_id is limited to only the last message?

Am I missing something? I’m looking for a way to extend this functionality such that I could, for example, pass a single parameter that would fetch the entire history of the conversation, not just the last message.

2

It seems to get the whole thing. I tried it just now:

Hello! This is a test of whether my script’s history is working. In a few turns, I’ll ask you what the magic word is (it’s banana). Don’t mention it until then!

Got it! I’ll wait for you to ask.

In the meantime, how are you today?

I’m just a program, but I’m here and ready to help! How are you doing?

Doing OK I guess.

I’m glad to hear that! If there’s anything you’d like to chat about or need help with, feel free to let me know.

What was the magic word?

The magic word is banana.

4 Likes
3

Interesting. I was testing last night and did not get this result. My test case was a little more convoluted though. I’ll have to try this out the way you did. Thank you!

1 Like
4

Here’s the code I used just in case.

from openai import OpenAI
api_key = "REDACTED"
client = OpenAI(api_key = api_key)
last_id = None
while True:
	user_input = input("\n> ")
	if user_input.lower() in ["exit", "quit"]:
		break
	if last_id:
		response = client.responses.create(model = "gpt-4o", input = user_input, previous_response_id = last_id)
	else:
		response = client.responses.create(model = "gpt-4o", input = user_input)
	last_id = response.id
	print("\n" + response.output_text)
1 Like
5

Thanks! I was able to replicate this on my end so indeed, it appears to be grabbing the whole history. Appreciate your assistance.

1 Like
6

Are all previous tokens in the conversation charged as input tokens in this case? (ie, same as manually managing state with the completions API)

2 Likes
7

What if I want not the whole history but only last n messages? How to do this?

2 Likes
8

Are all previous tokens in the conversation charged as input tokens in this case?

The docs say “Even when using previous_response_id, all previous input tokens for responses in the chain are billed as input tokens in the API.”

1 Like
9

I suppose eventually it should support truncation strategies similar to the Assistants API?

1 Like
10

Hey man thanks for responding in this thread.

Im trying to wrap my head around this, but struggling.

What i cant understand is the context of the input tokens being passed into each turn using previous_response_id.

for example, in my use case i am using file search in the assistants api. for every turn this passes chunks received from retrieval + user message.

now if in response api previous_response_id account for EVERY input, does that mean each retrieval input is all inputted. If so, that’s ridiculous.

Any help would be great please guys!

11

Ok so i think i get it, but It would be good to know how openai are handling it.

So if you want consistent memory as a thread.

  1. Initial call ( first message)
  2. get response (store the previous_response_id)
  3. create second call and add the previous_response_id.

and then so forth.

It must be a chain, so each turn you must give the last (previous_response_id).

It would be good to know how openai link all those responses together.

If i am completely honest, having a thread_id is a lot more simple. Can anyone see any benefits in the above workflow, rather than having a thread id.

Also it would be good to know if the “instructions” is used as tokens on each turn.

Also from testing i can see how this can get very messy in the api dashboard under logs. Its very difficult to follow a conversation.