Today i made couple of test using davinci-text-002 and davinci-text-003.
I am using Google App Script to send the requests to openai. So basically i am getting prompts from different cells and sending API request one by one…
Seems like there is huge difference in terms of response time.
Does anyone know what might be the reason? I mean, i am sending exact same prompts but 002 responding me in total of 24 seconds whereas 003 response time is 54 seconds. More than double. Any thoughts?
While I don’t work for OpenAI, and I don’t have inside details, I would guess the higher quality output (including longer) is responsible for the increased time. I’m sure it’s a balancing act…
Getting longer response times too for davinci on 003 over 002. It’s causing timeout limits on my serverless functions so while it’s producing better content it’s maybe not so useful because it’s taking so long to respond.
It would be good to get some update from the OpenAI team, I know it’s new so may be teething problems?
Davinci-003 provides a longer output on average as well. Did you factor this into your response times? Longer outputs generally take longer, so that could be part of it, in addition to higher quality outputs.
I’m having similar experiences with 003 when it comes to response times. Output is consistently better, but at times takes quite long to load. Then, other times, it’s as fast as 002. The length of the output does not really seem to be a factor here, sometimes it takes a few seconds even for a simply reply.
Is this known and intended? Or something that can and will be improved upon?
Just a few minutes ago I tried again, and this time 003 responded much faster! Similar to 002. Maybe something was fixed already? At least I know it can be fast now
// brief update: today I experienced some longer response times again. It seems to fluctuate (… based on load on their api? )
Maybe the large amount of trafic coming for ChatGPT (I read that they get to 1m users!) is taking a part of the processing power that is normally dedicated for instructGPT
Seems like that’s what’s happening. Just interesting that davinci-002 is not affected; apparently 003 and chatGPT share some infrastructure in that case.
My Article writer tool started showing Timeout error since last 2-3 days. Earlier it was generating awesome results on davinci-003. Any resolution to this apart from moving back to davinci-002?
P.S. Other tools working fine on davinci-003 however, being generating a longer output, the Article writer tool is experiencing Timeout.
error- Failed to load resource: the server responded with a status of 524 ()
Is anyone experiencing a much longer response time to your app? We have an app and the response time has gotten worse. Interestingly enough some of our competitor’s response time is faster. Any do’s and don’t to minimize the response time?
)