PDF file uploaded to vector store cannot be used by the model

I have 5 files uploaded to a vector store, all of which have the status completed. However when prompted to return all file names, it misses out one. Then after some debugging, I found out that the file has the usage_bytes=0, which I assume means the model skips over that file as it sees an empty file which is no use to it. But I cannot upload the file in a way which causes the usage bytes to not be 0. When I upload the file separately within the openAI playground, it has no problem at all reading it. The file size is relatively much larger than the others, at ~200KB compared to the second biggest at 35KB. The file in question is the attached image as a pdf

[Image with PII redacted by moderator]

Okay so the image was removed cause it thought the image contained real PII, the information was fake though

Hi and welcome to the community!

As a fast workaround to get you unblocked:
try sending the images inline as base64 encoded string.

This is something that has come up recently in a comparable context and should help until this is resolved.

I expect the issue you are running into is that the PDF contains an image only. OpenAI’s embedding models do not support image files. Wrapping an image inside a PDF does not return an error when the file is added to the vector store, but there is also nothing from the image that can be processed and retrieved through file_search later.

You can either send these files directly to a vision-capable model or extract the text first and then store that output in a vector store for file_search.

That makes sense and yes, the file is an image that is saved as a pdf. But how come if I upload the same file direct to the playground api does it then work as expected?

Using the playground chat pdf files are processed differently because on models with vision capabilities, the API extracts both text and page images and sends both to the model.

This file_inputs process is described here:

Do you have a method to distinguish pdf file contents before creating vector stores or how do you plan to work with this going forward?

Okay, I shifted my approach at this. I will just upload the files via the python script I’m making, rather than use the vector stores, as I don’t actually need it to remember the data. Thank you for your help :slight_smile:

Hey DanTheManIsBack, glad you found VeitB's suggestion helpful!

One small clarification for anyone who runs into this later: usage_bytes=0 on the vector store file doesn't necessarily mean the original PDF itself is empty. usage_bytes refers to the total vector store usage for that file, which may differ from the original file size.

Vector store files:

https://platform.openai.com/docs/api-reference/vector-stores-files

File inputs:

https://platform.openai.com/docs/quickstart

Avinash