Gradio app to classify image using gpt 4 vision

API
1

I am trying to create a simple gradio app that will allow me to upload an image from my local folder. The image will then be encoded to base64 and passed on the paylod of gpt4 vision api
i am creating the interface as:

iface = gr.Interface(process_image,"image","label")
iface.launch()

But I am unable to encode this image or use this image directly to call the chat completion api without errors

Has anyone tried?

I have however been able to create a FAstAPI endpoint which allows me to upload a file and return the expected output


app = FastAPI()

@app.post("/uploadfile")
def create_upload_file(file: UploadFile):
    name = file.filename
    type = file.content_type
    image_path = name
  # Function to encode the image
    def encode_image(image_path):
      with open(image_path, "rb") as image_file:
        return base64.b64encode(image_file.read()).decode('utf-8')

    # # Getting the base64 string
    base64_image = encode_image(image_path)

    headers = {
      "Content-Type": "application/json",
      "Authorization": f"Bearer {api_key}"
    }

    payload = {
      "model": "gpt-4-vision-preview",
      "messages": [
        {
          "role": "user",
          "content": [
            {
              "type": "text",
              "text": """ identify the following from the image: My prompt here
                            """
            },
            {
              "type": "image_url",
              "image_url": {
                "url": f"data:image/jpeg;base64,{base64_image}"
              }
            }
          ]
        }
      ],
      "max_tokens": 300
    }

    response = requests.post("https://api.openai.com/v1/chat/completions", headers=headers, json=payload)
    print(response.json()["choices"][0]["message"]["content"])
    # message = json_data["output"]["choices"][0]["message"]["content"]
    return {'status':'loaded successfully, check console','output': response.json()["choices"][0]["message"]["content"]}