Gpt-4-vision-preview very slow responses

API
, , ,
1

Anyone else having very slow responses or no response at all from gpt-4-vision-preview? I got it working the first few times, but it seems to be hanging now, no response even after 5 minutes.

const base64String = await downloadImage("https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg")


  const response = await openai.chat.completions.create({
    model: "gpt-4-vision-preview",
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: `What's in this image? ` },
          {
            type: "image_url",
            image_url: {
              "url": `data:image/jpeg;base64,${base64String}`,
              // url: "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
            },
          },
        ],
      },
    ],
    max_tokens: 4096,

  });

async function downloadImage(url: string): Promise<string> {
  const response = await fetch(url);
  const buffer = await response.arrayBuffer();
  const base64data = Buffer.from(buffer).toString('base64');
  return base64data;
}

Running in NodeJS

2

the API is working now, it was down for about an hour, tried again and it works!

3

How is the response time now? Are you getting response within 1 or 2 seconds?

4

Each image that I am using is 2048x2048 pixel and delay 20 sg
It’s not optimum

5

Each image that you are sending is resized to 768x768. Perhaps in multiple steps.

You can read carefully the API documentation of vision, especially regarding tiles and sizing, detail parameter, and do the heavy lifting of resize using your own compute.

You can use stream:true to see how fast models start generating response tokens.