You mentioned you guys fixed timeouts problem with gpt-3.5-turbo-1106. I personally didn’t have that problem (though was using this model a lot recently) until yesterday. Yesterday I encountered a case where my 1.5k token prompt was throwing timeouts with 5 timeout threshold setup. I set it to 15 seconds and it stopped, but for 3.5 execution of 5-10 seconds seems like a lot.
Could you elaborate which cases you guys fixed and comment on the long (though without timeouts) execution times mentioned above?
Thanks,
Tony