The first time this prompt was used, it correctly showed a list of results describing the events. All subsequent attempts have resulted in the response “I’m sorry, I can’t provide assistance with these requests.”.
This is not expected behavior.
`
PROMPT_MESSAGES = [
{
“role”: “user”,
“content”: [
{
“type”: “text”,
“text”: “These are frames of a video. Take a look at each frame and describe the events that are taking place.”
},
*[
{
“type”: “image_url”,
“image_url”: {
“url”: f"data:image/png;base64,{image}"
}
}
for image in base64Frames
],
{“type”: “text”, “text”: “detail: low”}
],
},
]
params = {
“model”: “gpt-4-vision-preview”,
“messages”: PROMPT_MESSAGES,
“max_tokens”: 2000,
}`
I have tried several variations of the prompt such as "“These are frames of a video. Describe the events in the frames.” and all attempts are resulting in the same error.
2 Likes
Out of curiosity, is it the same video/screengrabs giving you trouble? Maybe it notices it as copyrighted material or something? Can you reproduce with other videos?
Also, being so new, it might be down again. The status page lags a bit usually, but their team is usually on it rather quickly with so many customers clamoring…
I have only tested with on set of screenshots from a Mr. Beast youtube video.
To resolve the issue I did the following:
- Reduce the number of images in the request to 9. The LLM produced a useful response.
- Increase the number of images in the next requests to 19, followed by a request with 40 images, then 96 images.
I still don’t know why this problem began, or why reducing the number of images solved it.
I got the same problem here when I simply send one medical image to the API. It works for some images but for more than half of the images they reused to answer my question.
1 Like