Whisper API times out causing charge on retries

API
,
1

Hi,

I’m posting 15MB files to the audio transcription endpoint. It frequently times out and I still get charged.

conn = Faraday.new(faraday_options) do |f|
  f.request :multipart
  f.adapter :net_http
  f.headers['Content-Type'] = 'multipart\/form-data'
  f.headers['Authorization'] = "Bearer " + key
end

payload = { file: Faraday::Multipart::FilePart.new(file, fileinfo.fetch('content_type')),
            model: 'whisper-1',
            response_format: 'verbose_json' }
@response = conn.post(transcriptions_endpoint, payload)

I already set the timeout to 120 seconds, to give it a lot of time. It’s posting from an AWS so the connection speed shouldn’t be an issue.

The error is intermittent, so I know that the code works and the timeouts don’t always happen.

Does anyone have any suggestions other than splitting files down further (docs say up to 25MB is ok)?

Thanks,

Tom

2

Hi and welcome to the developer forum!

Be worth trying to replicating the setup from your own hardware and see if the timeous continue, i.e. run your code on your local machine.

3

Hi Foxabilo,

Thank you for your response.

It’s pretty much the same locally. I measured the time requests take. Some requests take 112 seconds, so go through when the timeout is set to 120 seconds. Some take more and fail.

I could just increase to 3 minutes or so, but not sure if that’s sensible.

4

I think the best option is to expand the timeouts to a range that covers 99.9% of typical operation, especially for high latency calls such as these. Really you are looking to catch errors in operation, being slow is not an error if the task is complex or the service is under load.

1 Like