Hello I prepare a data for finetune to davinci. i have my API Key. and I load data as jsonl with openai tools fine_tunes.prepare_data -f <LOCAL_FILE>. when I try to fine tune with openai api fine_tunes.create -t output.jsonl -m davinci. I get Stream interrupted (client disconnected).
To resume the stream, run:… when I check openai api fine_tunes.list I see my job in pending status. when I run openai api fine_tunes.follow -i id I get same issues. Anyone have any ideas on this?? Thanks.
Hi and welcome to the developer forum!
Fine-Tunes seem to be running very slowly at the moment with some users reporting having waiting over 20 hours for theirs to start processing. The stream interrupted message is not an error in the fine tune, it is simply a disconnection from the monitoring connection, using the command given you can reconnect and check on the status of the fine-tune job.
hey thank u for reply. when I check openai api fine_tunes.list for see statuses my Job in pending.
do you mean to say that the problem is on openai side and i should wait??
Correct, Pending means that the job is waiting to be processed.
@Foxalabs is the Fine-tune issue resolve ? when i run .list CLI i still get disconnect error and file is not in the queue
The disconnect is normal, so long as your fine-tune job gets processed, then it’s working normally, the disconnect thing is just the system disconnecting you from the monitoring connection you can always reconnect.
Yes, the issue should now be resolved, it may still take several hours for your job to start.
No, When i run openai api fine_tunes.follow -i it still says create fine-tune
And when i use <.list> it says pending. Its not going in the queue
Hi,
“pending” means it is in the que, pending it’s allocated slot to be ran.
actually last time also it was showing in pending for 2 days, and then when i ran the follow CLI , then only it went to queue
Ok, understood, hopefully the fix that was put in place earlier today means that it will run ok, could be worth starting another tune run in case something has got stuck.
@Foxalabs , so the fine-tune is down do you have any idea why i am getting these … in the response when i test fine-tune in playground.
Thank you
I don’t know what data was in your fine tune, I can’t comment on why the response looks like that.
what are your settings for that test?
check the attached image I guess i have very stupid settings ever. Can you help me fix it ?
I have such data in my fine-tune, your help would be greatful.
{“prompt”:“How does randomised algorithm work\n”,“completion”:" The algorithm typically uses uniformly random bits as an auxiliary input to guide its behavior, in the hope of achieving good performance in the "average case" over all possible choices of random bits .“}
{“prompt”:“what do you mean by bestfirst search\n”,“completion”:” bestfirst search is a search algorithm which explores a graph by expanding the most promising node chosen according to a specified rule.“}
{“prompt”:“how do you explain a daemon\n”,“completion”:” daemon disk and execution monitor is a process that runs in the background without users interaction they usually start at the booting time and terminate when the system is shut down.“}
{“prompt”:“What is phonetic algorithm\n”,“completion”:” A phonetic algorithm is an algorithm for indexing of words by their pronunciation .“}
{“prompt”:“what do you mean by uniform costsearch\n”,“completion”:” a tree search that finds the lowestcost route where costs vary."}
Ok, the fine-tune data looks fine, but it also seems that you are trying to teach the model new information, as in you are trying to teach it new data from a corpus of Q/A, that is not how fine-tunes work, the model will learn the style of the questions and answers, but not the data itself.
So you model will answer in the style of the person answering those questions, if you want to store actual data, you should look at embeddings OpenAI Platform
@Foxalabs , Basically i was testing with this fine-tune data. Tho my goal is completely different i am looking to create a ChatBot assistant for my SaaS product. which help customer with their general queries can you tell me the right path / steps to take ? I would appreciate your help
I’m happy to help with technical assistance and pest practices, but I work full time as a consultant, and I don’t have the spare time to help with an entire SaaS project. You can find many guides online, youtube, and other video sites, there are also excellent free courses over on