File Upload Example?

I was wondering if anyone had a working File Upload example in python or otherwise.
I would like to provide a large training file so that the completions API would be more accurate but from what I can tell the File Upload only works with the Search API. It would be good to have one or more working examples as starting points on how this is supposed to work and the proper format of the working JSON file.

Here is an example I tested for search:

import csv
import jsonlines
import json

def get_data_from_csv():
    with open('example-data.csv', encoding='utf-8', mode='r') as f:
        data = list(csv.reader(f, delimiter=";"))
        del data[0]  # remove header
        return data

def write_json_lines(data):
    with open('upload-files.JSONL', 'wb') as f:
        writer = jsonlines.Writer(f)
        for i in data:
            line = {"text": i[0], "metadata": i[1]}
            writer.write(line)

def main():
    data = get_data_from_csv()
    write_json_lines(data)

if __name__ == "__main__":
    main()

I suppose these work the same for all purposes. Not sure if it solves your problem but it produces a valid jsonlines file that can be processed. Hope that helps.

1 Like

This just looks like a CSV reader. All of the sample in the documentation are playground/completion samples and I was wondering if there was an end-to-end example/sample of either the Search or the Classification API endpoints.

Hi Greg, I think Kris was just sharing an example of how to make a working JSONL file. You can’t upload JSON files, but you can upload JSONL files. Here’s an example of uploading a JSONL file for Search:

openai.File.create(file=open("output.jsonl"), purpose='search')

And here’s an example of upload a file for Classification:

openai.File.create(file=open("output.jsonl"), purpose='classifications')

These JSONL files are now uploaded to OpenAI and can be used at any time. To add to that, you can see the full details for uploading a file here.

Hope that helps! Please let me know if you have any other questions!