Inconsistent follow up answer of assistants using API compared to Playground

API
1

I’m developing an application for non-streaming conversations with assistants via API, primarily utilizing files stored in a vector database. While everything seems to work fine initially, I’ve noticed an inconsistency in response behavior.

The first response to any question asked via the API is almost identical to the response generated in the Playground—exactly as expected. However, the issue arises with subsequent questions. The answers, though not incorrect, differ slightly between the API and Playground. Both responses cite the same source file, but the phrasing or details in the API-generated answer vary.

Interestingly, when I ask the second question in a new thread (both in the Playground and via API), the initial response matches perfectly again, just like the first question did.

I’ve ensured that all parameters—temperature, top-p, model, etc.—are identical for both the Playground and the API requests. Given this, I’m wondering if this could be a bug in v2 of the API. Many users have noted that the API should behave similarly, if not identically, to the Playground.

Has anyone else experienced this issue?

1 Like
2

Same issue here. More than one bot started changing his behaviour without any changes. Also, when i checked the assistant the name and model had changed by itself.

1 Like
3

So I rechecked everything and found that temperature = 0.01 is the only requirement for consistent and same response between playground and API request.

I had tested earlier and found the answers to be different, but after several more tests and with various other files attached as vectore store, I conclude that it is temperature setting only that matters.

Also found a further improvement in the answer matching between two when I add default_headers={“OpenAI-Beta”: “assistants=v2”} parameter while creating API Client.

1 Like