This is our first shot on goal of making a higher level API (which is still in Beta), future iterations will be much quicker. As it turns out, building something people will really love building with takes time a while. I am hopeful the wait will be worth it : )
17 Likes
Excited by your update on OpenAI’s beta API, Logan!
Any chance to join the waitlist
and contribute feedback?
Taking time to develop something like this, at this scale, is perfectly legitimate. I’d be happy to wait a few more months, if there was at least a hint of a timeline to work with.
The issue is with communication… streaming was announced as “coming soon” months ago, and the state hasn’t changed.
Timing right now is everything. I’ve been lurking for two months, and every day I wonder whether I should finally allocate a week+ of work to roll my own Assistant + streaming, or wait just another day… without knowing if “soon” is 1 more day, week, month.
3 Likes
Thank you for the additional information @logankilpatrick. As feedback from an implementation developer perspective, it would be really impactful for our capacity to use the Assistants API in production if it was more transparent in two key aspects:
- Internal Tool usage: For the documentation to explain how the assistant interacts with the tools retrieval, code interpreter, and dall-e, including the instructions, functions available, etc. This would ideally include a breakdown of how the Assistant does RAG. How are the documents indexed? which embedding model is used? What is the workflow for retrieval?
- Token usage: At the moment, this may be the biggest limiter we have seen from clients and other developers. There is little predictability in cost, which is particularly complex whilst using retrieval or code generation, which can sometimes un unpredictably use huge amounts of tokens are used by the browsing tool. Not only access to after-the fact counts of tokens, but also control oon max-tokens per thread/run + rate limits for assistants would be a great start.
Thank you and the team for all the work you do! Looking forward to seeing more
3 Likes
That made my day! Thanks for communicating, highly appreciated. Output streaming cannot come soon enough 
I’d like to second this comment. We could really use streaming as soon as you all can get it done!!
2 Likes
Thanks for the response, really glad to hear there is still progress being made and the next milestone is near.
Attention OpenAI, still waiting for Response Streaming. this is really stumping developers not being able to use Assistants API as a production ready option for building chatbots without having to worry about all the maintenance previously required.
Sora is great. But don;t forget us, your other revenue stream.
6 Likes
Any update here Logan? We have built a product around the API and are in a holding pattern waiting for streaming as it is completely unusable at the moment. Just an expected timeline would help a ton. Thanks, sir
3 Likes
Hey Logan, any updates yet? Thanks in advance
5 Likes
Unfortunately, Logan no longer works at OpenAI.
Hi all — we’re close to launching streaming and are beginning testing in the playground this week. We’ll let you know once it is live!
27 Likes
Thanks. This is great news. You have just changed our trajectory.
1 Like
Amazing, thanks!
Getting greedy here… but by any chance also vision implemented at the api? 
2 Likes
You can just use a function tool and use vision through the completions api. It’s what i’m doing for my assistant
2 Likes
Thank you for the update 
It looks like he Assistants playground keeps streaming the same response over and over again - unsure where to report this. Still a way to go, but I’m excited to see progress! 
Sorry about this! Could you share more details or a reproduction? Run ID / Thread ID would also be helpful.