Been using an MD and a TXT file on my Assistant, and both are failing
But the documentation does say that md and txt are allowed as per here: https://platform.openai.com/docs/assistants/tools/file-search/supported-files
Been using an MD and a TXT file on my Assistant, and both are failing
But the documentation does say that md and txt are allowed as per here: https://platform.openai.com/docs/assistants/tools/file-search/supported-files
Hey @idonotwritecode ,
ChatGPT automagically understands the markdown format in .txt files;
Rename your markdown files from .md to .txt
If you want to give ChatGPT a hand;
• At to the the top of the text file write: The following information is in Markdown format
• (or) Append (Markdown format) to the file names: e.g. “Information for my GPT (Markdown Format).txt”
Did that solve it?
This isn’t ChatGPT.
Its the Assistants API. Unfortunately, the error is that the file is not supported when it clearly says in their doc that it is.
Luckily the solution works for all OpenAi GPT-4 or higher models, regardless of end-point to the LLM.
Try it
vector store is not model
Hi!
You likely already did this, but can you confirm that the files you are trying to upload via the platform interface are not too large?
The maximum file size is 512 MB. Each file should contain no more than 5,000,000 tokens per file (computed automatically when you attach a file).
Have you tried uploading the files via the API directly to check if it is a general issue rather than one specific to the interface you are using?
Tried both.
Tried directly uploading it, and via API.s
When uploading via the platform.openai, you do not get any error. But when I do it via the API, I almost immediately get an error back.
I have raised a support case to OpenAI, but the SLA is 3 days.
I can confirm that this used to work before, as we use to upload a lot of txt files.
Tanveer
This might be due to a recent change in how the API functions, which may have already been accounted for when using the platform.
Could you please share the code for the API call you’re using to upload files? This will allow the community to review it further.
I use no code tools for this.
Here you go:
Upload the file using - https://api.openai.com/v1/files
API reference: https://platform.openai.com/docs/api-reference/files/create
My vector store is already generated per assistant as vector_id
Assign the file_id from step 1 using https://api.openai.com/v1/vector_stores/[vector_store]/files
API reference: https://platform.openai.com/docs/api-reference/vector-stores-files/createFile
Check this using a GET request to https://api.openai.com/v1/vector_stores/[vector_id]/files/[file-id]
API reference: https://platform.openai.com/docs/api-reference/vector-stores-files/getFile
The response that I get is:
{
“id”: “file-xxxxx”,
“object”: “vector_store.file”,
“usage_bytes”: 0,
“created_at”: 1725114837,
“vector_store_id”: “vs_xxxxxx”,
“status”: “in_progress”,
“last_error”: {
“code”: “unsupported_file”,
“message”: “The file type is not supported.”
},
“chunking_strategy”: {
“type”: “static”,
“static”: {
“max_chunk_size_tokens”: 800,
“chunk_overlap_tokens”: 400
}
}
}
Hey!
I quickly ran a test on my end using the V1 Assistant.
I can confirm that I was able to successfully upload both a .md and a .txt file and subsequently attach it to a vector store that I had created for that Assistant.
I used the same Python code as per the API specs.
These were just super small test files. I purely wanted to see if it works from a technical perspective and it did.
My best suggestion is to create another test case with your tools to check if they work as expected. Otherwise, you could consider contacting the creators of the tools for support, though I assume you’ve already done that.
In the meantime, you might consider quickly putting together a script to upload files.
Are you able to switch to v2?
Tried it on a few other API’s via Postman and getting the same error.
It’s late where I am. Happy to take another look tomorrow unless someone else from the Community has time before then to run another test.
It works the same, which makes sense given that the vector store is independent of the Assistant version. For some reason I thought earlier that you were using V1.
Thanks, @jr.2509, for confirming!
OP did link the v1 documentation above, but this might still be the issue.
The playground is using the v2 version of the Assistants API. OP confirmed that uploading files works in that environment.
There could potentially be a migration issue from v1 to v2. It’s worth checking if a newer or older version of the SDK might already solve the issue.
Otherwise, here is the link to the migration guide.
Since the v1 beta of the API will be deprecated by the end of the year, this is likely a type of question we will be encountering more often in the future.
Lastly, it is possible that we are looking at an error message that is not properly describing the underlying issue.
@idonotwritecode, please keep us updated on the actual solution.
So here is the rundown. This was an encoding issue (UTF-8).
Apparently, files that are UTF-8 saved (say as .txt files) can still be thrown as an error, if the encoding is broken. This can happen due to various reasons, such as a weird character, which you would normally find when scraping huge amounts of financial information (so think some formula’s, or weird characters).
Parts of the file could be uploaded without any issues. I did not think it was worth trying to figure out the exact character that caused my downfall, and rather added some simple REGEX filtering to my new scrapers to ensure it does not happen.
Thank you to those who gave good solutions + the OpenAI support team who confirmed this.