I’m testing retrieval in the Playground for the new Assistant API. I uploaded a 51MB file (JSON). When pressing save, I’m getting the following:
Failed to update assistant: UserError: Failed to index file: Error extracting text from file file-[redacted], detail: File contains too may tokens. Max allowed tokens per file is 2000000.
Is there such limits on tokens in files? Can’t find it documented, only seeing the 512MB file limit