Supported File formats Vector Store?

I’m currently working on vector stores and although I can upload .xlsx and csv as ‘normal’ files I cant seem to add those files to a vector store, is that correct?

If so, that would feel a bit odd, and I wonder if others have found other ways to get the content still indexed for semantic search? I was thinking of converting it into a supported format eg JSON, but I cant find anywhere a list of which formats are supported for semantic vector stores?

https://platform.openai.com/docs/api-reference/vector-stores
https://platform.openai.com/docs/api-reference/vector-stores-files