How to See the contents of OpenAI Fine Tuned Model Results in Python using the OpenAI API

I have fine-tuned a GPT-3 model for classification in Python. The model status still shows pending. However, the model events show that "Fine-tune succeeded. I even get an event showing the uploaded file id. When I download the results file using the following code and print its content, I see the following result:

content = openai.File.download("file-coumSxxxxxxxxxxxxxxxx")
contents = content.decode()


# Print the contents of the file
print(content)

Results:

b'{\n  "object": "file",\n  "id": "file-coumSxMyxxxxxxxxxxxxxxxx",\n  "purpose": "fine-tune-results",\n  "filename": "compiled_results.csv",\n  "bytes": 168364,\n  "created_at": 1673135506,\n  "status": "processed",\n  "status_details": null\n}\n'

Can somebody recommend how I can see the contents of the compiled_results.csv file?

I am also stuck because of the same problem. I have gone through the Open.AI documentation twice and referred to all the links W.R.T the topic. Also tried ChatGPT - No, answers. But I am sure that 1-month back openai.File.download() will directly download the Fine-Tuning results. OpenAI might have recently made some changes to that API.

Hi,
I think I found the answer. There are a couple of ways to get the fine-tuned result CSV. As far as I know 1. Python function (Like the one you have done), 2. Bash Commands, 3. API calls.
Using Python function and Bash commands I was not able to download the CSV file. Then I used a direct API call and it worked.
I use postman for API’s. GET https://api.openai.com/v1/files/file-xxxxxxxx/content.
Have Auth in the Headers. Hope it works for you.

hello, to get the file compiled_results.csv i used a GET request in POSTMAN. for me, it’s the best and easiest way to get the file.

using the ‘https://api.openai.com/v1/files/file-xxxxxxxx/content’ endpoint.
instead of ‘xxxxxxxxx’, you will place the ID of your fine-tuning, which can be obtained through the command (on colab, jupyter notebook or bash):

‘!openai api fine_tunes.results -i <model_fine_tuned_name>’

for auth: take your API key and place it in the POSTMAN authorization tab at: Authorization → Bearer Token → Token.
after sending the request you will be able to see the raw format file in the body of the request, to save it on your machine just click on ‘Save Response’

2 Likes

Hi, as of July 2nd, I can use your code to retrieve the content.

content = openai.File.download("file-coumSxxxxxxxxxxxxxxxx")
ans = content.decode()
with open('results.csv', 'w') as f:
    f.write(ans)

It works fine.

How to see the content in the newer version API?

content = client.files.content(result_file_id)

<openai._legacy_response.HttpxBinaryResponseContent object at 0x000002AA9FA7E960>

You can use one of the methods of the return object:

file = client.files.content(“file-8WxbhkRXPaPKsQNw4EwdQ3be”)
file.stream_to_file(“myfilename.txt”)

If you want to dump out binary bytes, you can use file.content. or file.text for a string.

You can only retrieve file contents on results files, such as those with purpose=‘assistants_output’. A list file call can get the file name the AI created.

Thanks for your reply But the content i got is something like below:

c3RlcCx0cmFpbl9sb3NzLHRyYWluX2FjY3VyYWN5LHZhbGlkX2xvc3MsdmFsaWRfbWVhbl90b2tlbl9hY2N1cmFjeQoxLDEuMDYxMDgsMC43NjE1OSwsCjIsMC41MDA5MiwwLjg3MjA1LCwKMywwLjkwODEzLDAuNzY5NDgsLAo0LDAuNzAzMDMsMC44MTA3MywsCjUsMC42MjIwNywwLjg0MDYyLCwKNiwwLjg0MTc3LDAuODA3NDMsLAo3LDAuNjA4MTQsMC44NDgzLCwKOCwwLjcyMDgzLDAuODQ4OTksLAo5LDAuNDg4OTEsMC44NzgwNSwsCjEwLDAuNzIzMzMsMC43OCwsCjExLDAuNTczNDksMC44NDc3NSwsCjEyLDAuNTcwOSwwLjgyODEyLCwKMTMsMC40ODExOSwwLjg3MDc1LCwKMTQsMC41MjY5OCwwLjg0NzY4LCwKMTUsMC41NDI2LDAuODQwODQsLAoxNiwwLjU1MTQ5LDAuODM2NzMsLAoxNywwLjQ5MTA2LDAuODcwMzEsLAoxOCwwLjU0NjU1LDAuODMyNzYsLAoxOSwwLjYxMTgsMC44Mzc5MiwsCjIwLDAuNjAyMjEsMC44MzA1NiwsCjIxLDAuNTExOTIsMC44NDczMSwsCjIyLDAuNTM2MTIsMC44NTYyMywsCjIzLDAuNTY0NTIsMC44MjkxMSwsCjI0LDAuNjU3MDcsMC44MjA4NSwsCjI1LDAuNTY3ODcsMC44NDE2MSwsCjI2LDAuNTQ1MTgsMC44NDA1MywsCjI3LDAuNjg4MjgsMC43OTg3LCwKMjgsMC43MDc1OCwwLjgwNDY0LCwKMjksMC40NTk1OCwwLjg4MjM1LCwKMzAsMC41MDkzMywwLjg1NDQzLCwKMzEsMC4zNjkzOCwwLjkyMDczLCwKMzIsMC40OTEzMiwwLjg0MjI3LCwKMzMsMC40NjIyMiwwLjg2NDg2LCwKMzQsMC41NTU4NSwwLjg1OTMzLCwKMzUsMC4zNzk1MywwLjg5MTE2LCwKMzYsMC40MjcyOSwwLjg5NDA0LCwKMzcsMC40MzA2NiwwLjg4ODU0LCwKMzgsMC40MTg0NCwwLjg5MjIyLCwKMzksMC41NjAyNiwwLjgyNjY3LCwKNDAsMC40NzI4NSwwLjg3MjIsLAo0MSwwLjQyOTAzLDAuODkzNzUsLAo0MiwwLjMzNzQsMC45MzYwMywsCjQzLDAuNTkxNDEsMC44MzA2MiwsCjQ0LDAuNTkwNDIsMC44NzE2MiwsCjQ1LDAuNTU3MjEsMC44NzI0OCwsCjQ2LDAuNDQ2NzksMC44ODczNywsCjQ3LDAuNTA4OTcsMC44NTQwNCwsCjQ4LDAuNDAxOTksMC45MDEwMiwsCjQ5LDAuNDQ0NjksMC44ODA5NSwsCjUwLDAuNDMyOTcsMC44ODQzNywsCg==

It is not a meanfully training loss info file.

here is my complete code after finetuning:
from openai import OpenAI
client = OpenAI()

job_id = ‘ftjob-b9S1AK4BHhBCYJv0I4LBZauM’

job = client.fine_tuning.jobs.retrieve(job_id)

response = client.fine_tuning.jobs.list_events(job_id)

events = response.data

events.reverse()

for event in events:

print(event.message)

result_file = job.result_files[0]

file = client.files.retrieve(result_file)

print(file)

content = client.files.content(file.id)

content.stream_to_file(‘myfilename.txt’)

It has meaning, just not one obvious to those that don’t recognize the pattern:

step,train_loss,train_accuracy,valid_loss,valid_mean_token_accuracy
1,1.06108,0.76159,,
2,0.50092,0.87205,,
3,0.90813,0.76948,,
4,0.70303,0.81073,,
5,0.62207,0.84062,,
6,0.84177,0.80743,,
7,0.60814,0.8483,,
8,0.72083,0.84899,,
9,0.48891,0.87805,,
10,0.72333,0.78,,
11,0.57349,0.84775,,
12,0.5709,0.82812,,
13,0.48119,0.87075,,
14,0.52698,0.84768,,
15,0.5426,0.84084,,
16,0.55149,0.83673,,
17,0.49106,0.87031,,
18,0.54655,0.83276,,
19,0.6118,0.83792,,
20,0.60221,0.83056,,
21,0.51192,0.84731,,
22,0.53612,0.85623,,
23,0.56452,0.82911,,
24,0.65707,0.82085,,
25,0.56787,0.84161,,
26,0.54518,0.84053,,
27,0.68828,0.7987,,
28,0.70758,0.80464,,
29,0.45958,0.88235,,
30,0.50933,0.85443,,
31,0.36938,0.92073,,
32,0.49132,0.84227,,
33,0.46222,0.86486,,
34,0.55585,0.85933,,
35,0.37953,0.89116,,
36,0.42729,0.89404,,
37,0.43066,0.88854,,
38,0.41844,0.89222,,
39,0.56026,0.82667,,
40,0.47285,0.8722,,
41,0.42903,0.89375,,
42,0.3374,0.93603,,
43,0.59141,0.83062,,
44,0.59042,0.87162,,
45,0.55721,0.87248,,
46,0.44679,0.88737,,
47,0.50897,0.85404,,
48,0.40199,0.90102,,
49,0.44469,0.88095,,
50,0.43297,0.88437,,

You have received a base64 response by the method used.

Which can be used to obtain more visual data.

Regular pointed me the correct direction - I was stuck for so long - The output is in a base64 format and you need to decode it

fine_tune_results = client.fine_tuning.jobs.retrieve(fine_tuning_job.id).result_files
result_file = client.files.retrieve(fine_tune_results[0])
content = client.files.content(result_file.id)
import base64
base64.b64decode(content.text.encode(“utf-8”))
To save content to file
with open(“result.csv”, “wb”) as f:
f.write(base64.b64decode(content.text.encode(“utf-8”)))

You can also get the output using GET request:
import requests
my_openai_key = os.environ.get(“OPENAI_API_KEY”)
auth_headers = {
“Authorization”: f"Bearer {my_openai_key}"
}
x = requests.get(‘URL’, headers = auth_headers)
print(x.status_code)

I hope it works for everyone!! Just decode that base64 output!

when was this change implemented? we had built a bunch of things on top of it and all of a sudden things broke