Hi, does anyone know if it is possible to use assistants API with the batching API? It would save us a lot of tokens. I couldn’t find anything in the docs or in the forum.
Hi welcome not sure how that would be possible since the batch API is just for chat competitions that don’t need near real time responses such as batch jobs.
Assistants API requires assistants to be ran on a thread which I don’t see support for in the batch API I don’t think they are compatible.
Does anyone know if OpenAI will make the Batch API also work with the Assistants API in the future, and if so, how soon that might come?
Hey @owainwilliams1 @bo1
What if someone provide this feature in langchain and you have option to use some open-source models also with lower rates as you are using batch-api? Will you move to the new async framework then?
I want the Batch API to support the Assistants endpoint to achieve lower gpt-4o token cost. Parallelism via Python is sufficient for me. As far as token cost, I know gpt-4o-mini is cheaper, but I prefer the increased accuracy of gpt-4o.
1 Like