(PHP) File upload for fine tuning not working [Solved, with tutorial]

API
1

Hello guys. I have been trying to fine tune a model but can’t get it to work.

So I tried the basics with a normal cURL completion and it worked perfectly, just like this:

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, “https://api.openai.com/v1/completions”);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, ‘{
“model”: “text-davinci-003”,
“prompt”: “Hello”,
“max_tokens”: 50,
“temperature”: 0.2,
“top_p”: 1,
“frequency_penalty”: 0,
“presence_penalty”: 0
}’);
curl_setopt($ch, CURLOPT_POST, 1);

$headers = array();
$headers = “Content-Type: application/json”;
$headers = “Authorization: Bearer sk-???”;
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);

$result = curl_exec($ch);
if (curl_errno($ch)) {
echo ‘Error:’ . curl_error($ch);
}
curl_close ($ch);

echo $result;

But when I tried to upload a .jsonl file for fine-tuning it just won’t work and I don’t really know what is wrong with the code.

$headers = array();
$headers = “Content-Type: multipart/form-data”;
$headers = “Authorization: Bearer sk-???”;

$c_file = “…/tests/chatgpt.jsonl”;

$opts = array(‘purpose’ => ‘fine-tune’, ‘file’ => $c_file);
$post_fields = json_encode($opts);

$ch = curl_init();

$curl_info = [
CURLOPT_URL => “https://api.openai.com/v1/files”,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_ENCODING => ‘’,
CURLOPT_MAXREDIRS => 10,
CURLOPT_TIMEOUT => 30,
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
CURLOPT_CUSTOMREQUEST => “POST”,
CURLOPT_POSTFIELDS => $post_fields,
CURLOPT_HTTPHEADER => $headers,
];

curl_setopt_array($ch, $curl_info);
$result = curl_exec($ch);
if (curl_errno($ch)) {
echo ‘Error:’ . curl_error($ch);
}
curl_close($ch);

echo $result;

I get this error:

{ “error”: { “message”: “‘file’ is a required property”, “type”: “invalid_request_error”, “param”: null, “code”: null } }

This is the chatgpt.jsonl file:

{“prompt”: “Prompt 1”, “completion”: “Completion 1”}
{“prompt”: “Prompt 2”, “completion”: “Completion 2”}

I have tried numerous ways but it just won’t work, please help me out.

1 Like
2

OpenAI API users cannot fine tune text-davinci-003

OpenAI API can fine-tune davinci (and a few other base models).

Also, you need to use the file API end-point to upload the file and get a file-id.

etc

HTH

3

Hello ruby_coder.

I did not specify a model on the upload file curl, so I don’t think that is the problem.

I used davinci-003 just for testing a simple completion and it worked just fine.

4

You cannot fine tune any OpenAI model named davinci-003. It is not possible.

Screenshot 2023-02-07 at 11.36.52 AM
Screenshot 2023-02-07 at 11.36.52 AM1624×368 56.6 KB

See:

OpenAI; Fine-Tuning

5

Please re-read my first post. There are two code samples, one with a completion using davinci-003 (works fine) and the second one with a file upload which is not using any model (not working).

Also, if you check the codes you will see I didn’t even try to fine-tune any model yet, because first I need to upload a file, and it is not working. So actually it is not the “fine-tuning” that is not working, but the file upload.

1 Like
6

You do not specify a model when you upload a file. There are only two params when uploaded a file prior to fine-tuning:

You must use the file endpoint and get a valid file-id before you proceed to the fine-tuning api.

HTH

Screenshot 2023-02-07 at 11.39.33 AM
Screenshot 2023-02-07 at 11.39.33 AM1274×614 52.3 KB

See:

OpenAI: File Upload

7

Sorry but are you reading my posts or just posting the first thing that comes to your mind? You are replying exactly what I just said.

2 Likes
8

Maybe it’s not recognizing the file because you’re sending the directory structure too? Try sending just the filename so OpenAI can check that the file once it’s uploaded…

Hope that helps.

9

I read your posts carefully.

Look at your errors, they are your errors, not mine :slight_smile:

{ “error”: { “message”: “‘file’ is a required property”, “type”: “invalid_request_error”, “param”: null, “code”: null } }

10

Hello Paul. Thank you for your answer, but I did try that and a lot of other file structres, none of them works. I always get the same error code.

11

Did you try just the file name?

Does it work okay from the command line with OpenAI installed?

12

It generally will not work with just the filename (unless you get lucky and are in the exact right dir path, so better to use the full path):

This works, for example, where I use the complete file system path:

 def self.upload(filename="#{Rails.root}/app/assets/files/fine-tune.jsonl",purpose='fine-tune')
        client = get_client
        response =client.files.upload(
            parameters: {
                    file: filename,
                    purpose: purpose,
                })
        file_id = JSON.parse(response.body)["id"]
        file_id 
    end
13

Yes, I tried with just “chatgpt.jsonl” and “@chatgpt.jsonl” and so many ways I can’t remember, none of them are working, and I made sure the filename is ok and that the file exists.

I did not install OpenAI, just using it through cURL. The cURL for a simple completion worked just fine.

14

Ah, yes, it would have to know locally where to read the file from … OpenAI prob strips it to the filename on their end anyway. Just trying to brainstorm…

I would try it from the command line with the python package…

15

Yeah, PHP is not how I would do this; as it’s easy in Python and Ruby.

I don’t think the OP is going to listen to that suggestion :slight_smile:

For those of us who have been though this before, it’s important to get the file-id first and make sure it is working before worrying about fine-tuning.

1 Like
16

There is a reason I asked for PHP in the title, that is the only way I can do it as the code will be on a virtual server that only runs on PHP and doesn’t have any other options like a command line or whatever.

And yes, you finally got it right, I’m just trying to upload a file first, not trying to fine-tune anything yet.

17

Huh? What are you trying to do exactly? Fine-tune once or set-up a fine-tuning service?

If a server runs PHP, it can likely run python too…

18

I need to set up a script in a virtual PHP server that will get the information from that server database and will update more than once, so like I said, other ways of doing it are not an option.

19

Actually I got it all right.

You @pezcri said you were able to fine-tune davinci-003 and this is totally wrong.

I have it right, you are the one with all the errors. My code works flawlessly and I have been though this debugging before and am trying to help you; but you seem “all over the map” to me.

Don’t shoot the messenger.

:slight_smile:

20

You might want to re-read all my posts again as I never said I was using davinci-003 for fine-tuning. I said I made a simple completion request on davinci-003 and it worked just fine.