GPT-4o Vision help Image input not working

API
1

I need my program to take a screenshot of the screen and then upload that into gpt4(I would like to use gpt4o if possible) , search the internet and then give me a response based off what its sees. But I cant seem to figure out the input for vision. Below is my code and the error that appears . Help would be appreciated

image_2024-06-17_081051385
image_2024-06-17_0810513851873×66 3.29 KB

import keyboard
import pyautogui
import requests
import pyperclip
import io
import base64

def take_screenshot_and_process():
    # Take a screenshot
    screenshot = pyautogui.screenshot()
    buffer = io.BytesIO()
    screenshot.save(buffer, format='PNG')
    image_base64 = base64.b64encode(buffer.getvalue()).decode('utf-8')

    # Prepare the API request
    headers = {
        'Authorization': 'Bearer key'
        'Content-Type': 'application/json',
    }
    data = {
        "prompt": "Describe this image",
        "model": "gpt-4",
        "n": 1,
        "size": "1920x1080",
        "image": image_base64
    }
    
    # Send the request
    response = requests.post('https://api.openai.com/v1/images/generations', json=data, headers=headers)
    response_data = response.json()
2

There’s a few issues with how you’re currently trying to call the API, I highly recommend reading the documentation since it clears up a lot of confusion.

That said, below is a good starting point that you can use.


    headers = {
        'Content-Type': 'application/json',
        'Authorization': f"Bearer {os.environ.get('OPENAI_API_KEY')}"
    }

    print("sending request")
    payload = {
        'model': 'gpt-4o',
        'messages': [
            {
                'role': 'system',
                'content': 'You are a helpful assistant. You will fulfil the user's requests to the best of your ability.'
            },
            {
                'role': 'user',
                'content': [
                    {'type': 'text', 'text': 'Describe this image'},
                    {'type': 'image_url', 'image_url': {'url': f'data:image/jpeg;base64,{image_base64}'}}
                ]
            }
        ],
        'max_tokens': 800
    }


    response = requests.post('https://api.openai.com/v1/chat/completions', headers=headers, json=payload)

    text = response.json()['choices'][0]['message']['content']
    print(text)
1 Like