After several hours file still being processed and is not ready to be used for fine-tuning

In my previous attempts all the files I uploaded for gpt3.5 fine-tuning were uploaded promptly and could be used to start the job after several minutes.

Yesterday I uploaded significantly larger file (still not very big - about 13500 examples) and after 9 hours when i try to create the tuning job I still receive

“error”: {
“message”: “File ‘{my_filename}’ is still being processed and is not ready to be used for fine-tuning. Please try again later.”,
“type”: “invalid_request_error”,
“param”: null,
“code”: “invalid_file_status”

the file itself has status that looks fine to me
“object”: “file”,
“id”: “…”,
“purpose”: “fine-tune”,
“filename”: “…”,
“bytes”: 44039452,
“created_at”: 1693518755,
“status”: “uploaded”,
“status_details”: null

Should I keep on waiting? If so, for how long?
Is there a limit on file size that can be used for training?


I resolved the issue, and leave the hint for future generations :slight_smile: → so I finally decided to re-upload the file, and when I did, it showed that there were issues with jsonl formatting (during initial upload, instead of returning the error, uploader just hanged - so it seems it is not deterministic). After solving formatting issues and another re-upload, everything worked promptly as usually.

@kabar can you share what was the formatting error in your file? I’m having the same error but the validation script suggested on github. com/openai/openai-cookbook/blob/main/examples/Chat_finetuning_data_prep.ipynb from OpenAI does not give me any error and the status return by openai.File.create says uploaded

For me the error occurred because of two reason, first the format of my JSONL and second one is
I gave my api key as

later I noticed that I need to do
openai.api_key=“mykey” to authenticate