Hi all,
Does anyone know when they will improve the process speed of the Assistant API? At the moment it is way too slow to use in production.
Thanks in advance!
1 Like
The current offering is a beta test to gather feedback and real world usage data, more features and refinement will be built into the assistants API with time.
You can help with this by reading the documentation in the resources section, posting Bugs in the Bugs category under API and posting Feedback under the Bugs Category, all available in the lefthand sidebar.
2 Likes
Thanks for the reply! I understand it is Beta, was just wondering if there was something of a roadmap or timeline on what to expect when 
1 Like
There are the DevDay videos and the documentation to follow, but there is no official timeline for what assistants will become as that is up to the developers.
1 Like
You describe correctly. The system is already making its calls internally with minimum network latency, so there’s no little “tweak”.
AI language models have a limited token generation rate, multiple unseen language model calls can be used within an assistant run, the final output cannot be streamed (because the AI still might be generating internal function calls), and that one must in code “check back” to see that you have an answer. Thus, that gives a product that indeed is not something to “chat with”, but “tune in later to see what the AI came up with”.
The only fixing by reprogramming is what OpenAI can do, a “developer” of an assistant must allow the agent to run autonomously and do whatever it comes up with. Changes could be to cap the iterations or improve task efficiency, but it seems nothing has been done at all in this month to improve on what was presented.
2 Likes
As this topic has an accepted solution, closing topic.