After several hours file still being processed and is not ready to be used for fine-tuning

API
1

Hi!
In my previous attempts all the files I uploaded for gpt3.5 fine-tuning were uploaded promptly and could be used to start the job after several minutes.

Yesterday I uploaded significantly larger file (still not very big - about 13500 examples) and after 9 hours when i try to create the tuning job I still receive

{
“error”: {
“message”: “File ‘{my_filename}’ is still being processed and is not ready to be used for fine-tuning. Please try again later.”,
“type”: “invalid_request_error”,
“param”: null,
“code”: “invalid_file_status”
}
}

the file itself has status that looks fine to me
{
“object”: “file”,
“id”: “…”,
“purpose”: “fine-tune”,
“filename”: “…”,
“bytes”: 44039452,
“created_at”: 1693518755,
“status”: “uploaded”,
“status_details”: null
}

Should I keep on waiting? If so, for how long?
Is there a limit on file size that can be used for training?

Regards,
Kabar

2

I resolved the issue, and leave the hint for future generations :slight_smile: → so I finally decided to re-upload the file, and when I did, it showed that there were issues with jsonl formatting (during initial upload, instead of returning the error, uploader just hanged - so it seems it is not deterministic). After solving formatting issues and another re-upload, everything worked promptly as usually.

3

@kabar can you share what was the formatting error in your file? I’m having the same error but the validation script suggested on github. com/openai/openai-cookbook/blob/main/examples/Chat_finetuning_data_prep.ipynb from OpenAI does not give me any error and the status return by openai.File.create says uploaded

4

For me the error occurred because of two reason, first the format of my JSONL and second one is
I gave my api key as
api_key=“mykey”

later I noticed that I need to do
openai.api_key=“mykey” to authenticate