Batch runs with Assistant starts hallucinating

API Bugs
1

Hi
I created an Assistant to clean up html code from a previous Wordpress editor and at the same time add an Elementor shortcode.
With Make i wanted to run it through a couple of hundred posts.
But after 3-4 posts the assistant started to invert some letters of the shortcode or add characters, making it useless.

What is happening here?
This makes an automation basically useless.

2

I infer from what you write:

  • You are using the Assistants endpoint, instead of Chat Completions
  • That the performance gets worse not randomly, but the more “work” that is being done.

From that I conclude:

  • You have a single-shot job, but are inappropriately using the conversational “Assistants” endpoint to do that;
  • That you may be adding more messages to a single thread, thus growing a conversation with every turn

I would use chat completions to script this, using messages in this manner:

system: “you are an AI with a single automatic API task and no user to talk to, which is …”
user: perform {task}, processing this data: {data}
response_format: (your strict JSON schema, with fields to be completed by AI that match what it is supposed to produce)

Thus, there is no implied conversation happening, and no growing cost of having a memory of that conversation. There is just a controlled input and output.

1 Like
3

I thought, by not adding the thread ID it would avoid adding to that conversation. But apparently not.
Thanks a lot!!