Managing timeout when waiting for the response from chat completions request

API
1

I tried to get chat completion (gpt-4) for a prompt that is expected to return a long reply:

async function askGPT(messages) {
  const completion = await openai.createChatCompletion({
    model: 'gpt-4',
    messages: messages
  });
  return completion.data.choices;
}

console.log(await askGPT([
  {
    role: 'user',
    content: 'Give me a list of 1000 random words, separated by commas'
  }
]);

This call died with the following error:

/myDevDir/node_modules/axios/lib/core/createError.js:16
  var error = new Error(message);
              ^

Error: Request failed with status code 524
    at createError (/myDevDir/node_modules/axios/lib/core/createError.js:16:15)
    at settle (/myDevDir/node_modules/axios/lib/core/settle.js:17:12)
    at IncomingMessage.handleStreamEnd (/myDevDir/node_modules/axios/lib/adapters/http.js:322:11)
    at IncomingMessage.emit (node:events:523:35)
    at endReadableNT (node:internal/streams/readable:1367:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {

I’m not sure what exactly the status code 524 informs about here. I’d like to know

  • if there’s a way to know for sure what was the cause of error: timeout or some other issue;
  • if there’s a way to salvage whatever has been received up to the point when the error happened, assuming there was a stream of tokens that went dead. Most of the time, we’d prefer to receive from a query at least something instead of nothing.
2

this may not be direct answer but exponential backoff kind of relieved my ratelimit and timeout error…

1 Like