Does anyone know when openAI will support images in vector stores? I’m getting this error when I try to attach an uploaded png file to a vector store, which per their documentation is expected at the moment:
400 Files with extensions [.png] are not supported for retrieval. See https://platform.openai.com/docs/assistants/tools/file-search/supported-files
That is correct, images are not supported at the moment. Here’s the full list of formats that are.
Is there are a particular reason you want to use the vector store for the images? Is the envisioned user flow one where the user asks questions about the contents of the images?
One potential workaround could be to embed images in a file whose format is supported at the moment. Candidates include .pdf, .pptx, .docx, etc. Not sure if this will work as intended, but certainly worth a try, I think.
Another workaround is using the vision capabilities first to generate some sort of descriptive text (e.g., captions). Since .pdfs are supported, maybe you can have an image and the associated generated text underneath it. I’m hoping the text section will be embedded successfully, and any application you build on top will at least reference the right file in this fashion.
Maybe someone can correct me, but if OpenAI is using their Embedding models for the purposes of converting files into numerical representations for the vector stores, then I think only text will be available for the forseeable future. A different image-to-embedding model would be required to create a numerical representation fit for a vector stores, and I don’t see any information about this online.