How can i input images and files to openai api so that it can read images and files and generate responses. also save the images and files into threads

API
1

Hi there, I am new to the community. I am stuck on adding images and files to the Openai API as input so that it can read images and files and generate responses, as well as save the images and files into threads. How can I do it?

1 Like
2

This is an excellent question.

3

You can use the assistants api platform. The endpoints are well documented. for example individual files can be up to 512 MB, and the size of all files uploaded by one organization can be up to 100 GB. The Assistants API supports files up to 2 million tokens and of specific file types. from the openai assistants api documents

Upload a file with an “assistants” purpose

file = client.files.create(
file=open(“mydata.csv”, “rb”),
purpose=‘assistants’
)

Create an assistant using the file ID

assistant = client.beta.assistants.create(
instructions=“You are a personal math tutor. When asked a math question, write and run code to answer the question.”,
model=“gpt-4o”,
tools=[{“type”: “code_interpreter”}],
tool_resources={
“code_interpreter”: {
“file_ids”: [file.id]
}
}
)

Files can also be passed at the Thread level. These files are only accessible in the specific Thread. Upload the File using the File upload endpoint and then pass the File ID as part of the Message creation request

thread = client.beta.threads.create(
messages=[
{
“role”: “user”,
“content”: “I need to solve the equation 3x + 11 = 14. Can you help me?”,
“attachments”: [
{
“file_id”: file.id,
“tools”: [{“type”: “code_interpreter”}]
}
]
}
]
)

Images can be managed by designating purpose as “vision”

file = client.files.create(
  file=open("myimage.png", "rb"),
  purpose="vision"
)
thread = client.beta.threads.create(
  messages=[
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What is the difference between these images?"
        },
        {
          "type": "image_url",
          "image_url": {"url": "https://example.com/image.png"}
        },