I was buzzing with excitement to dive into all the new features, but it wasn’t long before I hit a couple of major snags that kept me from really enjoying everything the way I had hoped.
From what I’ve gathered, there are a couple of key elements that need to be in place for anyone to even consider weaving the Assistant API into their setup:
1. Temperature
Getting the temperature right is crucial for consistent responses. Oddly enough, OpenAI didn’t include temperature controls, which means the responses can be a bit unpredictable (I suspect they put it on ‘1’). I usually find that setting the temperature somewhere between 0.3-0.4, occasionally even a bit lower, works best for me.
2. Streaming
It’s no secret they’re on it – they promised it would be available “soon.” Yet, here we are, four months later, and still waiting. Without streaming, it’s pretty tough to integrate any of this into a real-world application. Let’s be honest, nobody wants to stare at a loading icon for ages.
I’m left wondering when we’ll see a real update to the beta – if we get one at all, at this point.
Does anyone have the scoop on what to expect and when it might finally land?
Sure could be good - although I feel that right now I can ‘prompt’ the temperatue quite a bit (‘do not speculate, do not add your own conclusions etc’)
While I think streaming is cool - there are tons of ‘real world’ applications’ that do not need streaming at all. I process dozens of threads a day internally that require 0 streaming. Processing emails, incoming applications, data request, charts - I have about 30 different Assistants humming along - 0 streaming needed. And as these Assistants are getting to do bigger and bigger tasks (including calling each other) - ‘streaming’ is less important for that. So while there certainly is a usecase for streaming Assistants - I am not sure that in a year from now most Asssistant applications will ever be using streaming.
Regarding Temperature, I couldn’t agree more. I’m not sure how Assistants will actually be able to achieve anything close to their obvious objectives in production (i.e. deliver specific and customisable process outcomes) if they cannot be reliably configured as per Completions.
I am mothballing my current project while I wait for this basic capability to ship.
I’d love some feedback from regarding whether or not Assistant Temperature will happen, or whether moving back to Completions for better outcome assurance will be the only option.
—Wonderful news regarding the inclusion of Temperature in Assistants. Incredibly helpful and thanks for keeping us up to date.
Now, any chance we can get “seed” and “system fingerprint” params as well??
Thanks!
