At my use case, I isolated the problem with files that were exported from Google Drive. Probably something weird regarding wrong encoding…
If I created a plain new csv file with the same data at vscode and saved it, the import would work.
A workaround for those who want to use Retrieval feature and have files at Google Drive as your source: Export your files as .pdf instead of .csv.
My assistant is up and running with 10+ .pdf files that have previously failed as .csv imports. Hope that it helps 