[SOLVED] File upload REST API help?

[elsiokun]

Hi there,

I have built a simple web app (using Angular) to create illustrated stories for my kids using a simple form that creates prompts and retrieves text from Davinci and an image from Dall-E using the API.

I am keeping the best, edited versions of the Completions as a JSONL file somewhere online on my server.

I am now trying to build an admin page of this web app to upload, manage, delete training files and start fine tuning jobs, however I am unable to make the ‘file upload’ (POST method on the “file” endpoint) to work.

I am passing the JSONL file URL on my server as the “file” parameter of the request but I keep getting “400 OK” error response code with the following message : “‘file’ is a required property”

here is the request payload

{"file":"https://japansio.info/api/training_data.jsonl","purpose":"fine-tune"}

I have tried with an “@” as in the documentation example but I’m guessing I’m missing something bigger like the fact that I should probably pass the “content” of the file to the endpoint rather than it’s location ? But I have no idea how.

Any help would be appreciated

[raymonddavey]

The file has to be uploaded using the file end point

It will give you an ID that you use instead of the filename

Then you trigger the processing of the file you have uploaded using the ID

[elsiokun]

thank you for your reply but my problem is that I can’t make the upload work…
I am passing the file’s URL on my server as the ‘file’, parameter but it’s not taken into account

right now I’m trying to get it as a string and stream it using form data but I’m running circles here

[raymonddavey]

I understand now

you need to send the file as a byte stream. Tell me the programming language you are using and I will see if I have some sample code that will help

Here is a Python example

openai.File.create(
file=open(“mydata.jsonl”, “rb”),
purpose=‘fine-tune’
)

Then you might back

file-XGinujblHPwGLSztz8cPS8XY

as the id

Then you call

openai.FineTune.create(
training_file = “file-XGinujblHPwGLSztz8cPS8XY”,
model = “ada”,
suffix = “chatbot”,
)

and you get back

ada:ft-your-org:chatbot-2022-12-18-07-47-04

(or similar)

[elsiokun]

I’m using javascript/typescript (angular app).

[raymonddavey]

This is some “Typescript” code for Dall-E. It uses files as well

const response = await openai.createImageEdit(
fs.createReadStream(“sunlit_lounge.png”) as any,
fs.createReadStream(“mask.png”) as any,
“A sunlit indoor lounge area with a pool containing a flamingo”,
1,
“1024x1024”
);

The key thing is that they are using a file system ReadStream (but it is an image). I suspect that if you are not using an openai library to help you, you will have to read the file at your end and feed its content instead of the filename.

[elsiokun]

Thank you for your help but i’m still struggling.

The content of file I wish to upload is entirely contained in a string object that I try to send as a Blob using the following code

  UploadFile(token: string, file: string): Observable<TrainingFiles> {
    let fileblob = new Blob([file], {
      type: 'application/jsonl',
    });

    let body = new FormData();

    body.append('purpose', 'fine-tune');
    body.append('file', fileblob, 'myTrainingFile.jsonl');

    let httpOptions = {
      headers: new HttpHeaders({
        'Content-Type': 'multipart/form-data',
        Authorization: 'Bearer ' + token,
      }),
    };

    return this.http.post<TrainingFiles>(
      'https://api.openai.com/v1/files',
      body,
      httpOptions
    );
  }

Which changes the request Payload to this :

…
Content-Disposition: form-data; name=“purpose”

fine-tune
------WebKitFormBoundary4nWKI7Szoubzeqlg
Content-Disposition: form-data; name=“file”; filename=“myTrainingFile.jsonl”
Content-Type: application/jsonl

{“prompt”:"raconte moi une histoire pour enfant dont les héros sont fifi la girafe et rhino le rhinocéros et qui vont à la montagne pour faire du parapente avec le
…

I think I’m kind of close but no idea why the error is still “‘file’ is a requirede property”

[elsiokun]

I have posted the question in Stack Overflow as it doesn’t seem an OpenAI specific question (the Upload endpoint works fine from postman) but more a “client side implementation” of the endpoint.

If interested about the topic you can follow the question here at stackoverflow

Thank you.

[elsiokun]

Update : the issue was solved by removing the ‘Content-Type’: ‘multipart/form-data’ line in the above code.

Here is a working implementation :

UploadFile(token: string, file: string): Observable<TrainingFiles> {
    let fileblob = new Blob([file], {
      type: 'text/plain; charset=utf8',
    });

    let body = new FormData();

    body.append('purpose', 'fine-tune');
    body.append('file', fileblob, 'myTrainingFile.jsonl');

    let httpOptions = {
      headers: new HttpHeaders({
        Authorization: 'Bearer ' + token,
      }),
    };

    return this.http.post<TrainingFiles>(
      'https://api.openai.com/v1/files',
      body,
      httpOptions
    );
  }

[PaulBellow]

Nice! Thanks for coming back to share with us.

[noah.s1129092]

The exact same problem I had, not adding anything solved it.

[unispode]

Test code works in Browser console:

const formData = new FormData();
let j1 = {‘prompt’:‘what’s the openai rest api url of file upload’,‘completion’:‘https://api.openai.com/v1/files’};
formData.append(‘purpose’, ‘fine-tune’);
formData.append(‘file’,new Blob([JSON.stringify(j1)+‘\n’],{ type: ‘text/plain’ }), ‘myTrainingFile.jsonl’);

fetch(‘https://api.openai.com/v1/files’, {
method: ‘POST’,
headers: {‘Authorization’: Bearer sk-......},
body:formData

}).then(res=>{console.log(JSON.parse(res))})