API BUG when call gpt-4-vision-preview

API
1

I use stream = true when I invoke the api to create chat completion.

I always got finish_reason = stop from chunk object.

But I don’t get the stop signature when I use model = gpt-4-vision-preview.

Anyone get the same issue?

2

The latest update has broken a lot of my apps using stream.

Have you tried checking the payload for “[DONE]” ?

Edit: If you have recently updated to v4 of the api, check your logic for where you check the finish_reason, they changed it slightly. It’s now

      for await (const chunk of response) {
        console.log(chunk.choices[0]);

        const payload = chunk.choices[0];

        if (chunk.choices[0].finish_reason === "stop") {
          res.end();
          return;
        }

.....
3

I noticed that the api is changed. I am using Java language and OPENAI JAVA SDK which is built by myself.

I tried to check the payload for “[DONE]”. And it is returned by API when using model gpt-4-vision-preview.

Now I use the event EventDone onEventDone to check it temporarily.

It is called around Line 217 in OpenAiService.java

@Override
public void onEvent(@NonNull EventSource eventSource, @Nullable String id, @Nullable String type, @NonNull String data) {
    if (data.equals("[DONE]")) {
        listener.onEventDone(requestId);
    } else {
        try {
            ChatCompletion chatCompletion = defaultObjectMapper().readValue(data, ChatCompletion.class);
            listener.onEvent(requestId, chatCompletion);
        } catch (JsonProcessingException e) {
            throw new RuntimeException(e);
        }
    }
}
4

I too have not been able to get GPT-4-vision-preview to work with streaming, so if you figured it out or do in the future, and don’t mind sharing, please let me know.

Honestly, the API documentation took a nose dive with the DevDay releases, which is understandable but disappointing, because I’ve always considered it maybe the highest quality documentation I’ve used for anything. Before DevDay, I can’t think of a single thing that ever confused me or that I couldn’t find on the docs.

But GPT-V sharing the chat completion api reference, and the request body definitions not even changing when you switch to it personally cost me several hours of tinkering today, and I wasn’t even able to get streaming working, just assumed that GPTV wasn’t able to do streaming after an hour and half beating my head against the wall assuming I must be doing something wrong.

Same with the modified JSON structure for messages, even putting its poor naming structure aside, to get that data you have to move from the api reference to the vision docs to get any explanation whatsoever on the changes.

Now I’m having a problem where it’s not producing enough text as a response, it’s definitely responding and working right, but it cut off in the middle. I think I’ve narrowed it down to max_tokens being a requirement for GPTV, I don’t know I’m going to test in a few minutes, but if that’s the case, it’d be nice for the API reference to say something about this.

I’m sure it will all get resolved, but in the mean time it is taking up a lot of dev time.

5

If your using js/ts I can fix your code for you if you’d like.

6

It’s api’s problem. And I’ve fixed around the issue with api.