API 504s in production (vercel) only

API
8

There’s a couple issues thats happening here:

  1. I’m assuming most people here are using the sample /api/generate scripts from the openai-quickstart-node project. It has the error handling in the wrong place. It should come before you try to interpret the response json on the line above.
    Unexpected token is throwing because you are trying to infer json from the text “An error occurred…”
    Move it down, drop the data.error, and only use the response.status in your Error like this
if (response.status !== 200) {
  throw new Error(`Request failed with status ${response.status}`);
}
  1. The unexpected error (504) is a timeout error. This is happening because Vercel deployment environments default to 10 second timeouts. This is an issue when using APIs like OpenAI that can take up to 12-15 seconds to fulfill a request.
    In order to increase this timeout, you must setup a custom vercel.json.
    This is very easy. Its just a file titled vercel.json in your project’s root folder, but there are a few more gotchas.

  • You need to have a Pro account in order to set a custom timeout or you will see an error like this

    Screen Shot 2023-02-02 at 7.58.08 PM
    Screen Shot 2023-02-02 at 7.58.08 PM1132×58 29.1 KB

  • Even after upgrading to Pro, your pushes may not automatically trigger a deployment because you’re adding a vercel.json file

Pushed commits will deploy automatically for public repositories. However, if the vercel.json file changed or the project has Environment Variables, a Team Member on Vercel will have to authorize the Deployment. This is a security measure that ensures changes to Environment Variables and other configuration properties are reviewed before a Deployment is created (it can be disabled if you prefer via the git-fork-protection setting). A link to authorize the Deployment will be posted as a comment on the Pull Request.

I disabled git-fork-protection in attempt to get around this, but it still wouldn’t automatically deploy. I ended up needing to install the vercel CLI and deploy it from the CLI manually.

After all was said and done, the random 504 error stopped occurring in prod and everything worked as expected.

See the follow up comment for the supporting links…2 link limit per comment for new users

4 Likes