I’d like to be able to provide a prompt (e.g. “George Washington”) and receive a JSON object back with various details (e.g. {“firstName”: “George”, “lastName”: “Washington”, “yearOfBirth”: 1732, “yearOfDeath”: 1799, “bio”: “First President of the United States”}).
ChatGPT seems to handle this well. I’ve tried fine-tuning GPT-3 to mixed results. Thinking this might be better suited for Codex but unaware of how to fine-tune that model. Curious as to what would be some best practices for this use case.
I’ve been trying to do the same using GPT-3 and prompting with a sample has helped a lot. The return from the API is a string format, which can easily be read as a JSON using json.loads if you’re using python.
This is easy to do with the davinci base model. You cannot fine-tune any codex models at this time.
I releases a tutorial / lab experiment on how to fine-tune a single-line JSONL training file using the n_epochs parameter yesterday. You might find the results useful.
Can you provide sample training data? I’m trying to figure out how to train my model with sample json and the api doesn’t seem to like json as a property inside of the overall jsonL file.
Yeah, exactly. Include a reference JSON object in your prompt (or as many as you can fit) and it should conform to that. It might forget after a while, so you may need to periodically refresh context.