We’re attempting to teach GPT4 to draw diagrams similar to UML. Problem is that our language is quite a bit different than UML. We do economy design and we have nodes that move resources in an inter-connected system. We expect a JSON output and we use JSON schema to constrain the answer to an array of elements.
All fine and dandy. Until GPT starts to connect Sources to other Sources instead of Pools or Drains, and a whole other bunch of illegal / non-sensical operations for our system. Of course, we give GPT a starting prompt which defines the rules of our system, but the prompt is quite large and GPT stumbles in the dozens of rules we throw in.
To fine-tune it, I suppose we should be sending a examples of valid JSON structures, correct?
Or should we send the text-based rules? (i.e.: “You cannot connect a Source to another Source”).
Or both?
You would ideally send it the corrected versions, not text-based rules.
The idea is that the model would implicitly understand the rules.
@RonaldGRuckus thank you for answering! 
.
One more question: as we all know, the format for training is comprised of sets of data that are formatted like this:
{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the capital of France?"}, {"role": "assistant", "content": "Paris, as if everyone doesn't know that already."}]}
So we have to have the triad of SYSTEM / USER / ASSISTANT.
While the System prompt is easy to figure out (it’s always going to be “you’re going to design Machinations diagrams for us”), and the Assistant answer is also easy (the actual JSON), the problem is the USER prompt.
What should I give into the User prompt? I probably should explain what the given JSON is actually doing, right? Problem is, right now, we have thousands of diagrams but not described in great detail. So it’s going to be pretty tricky to come up with a good user prompt that generates a certain diagram.
You want to provide examples of a typical conversation.
Your user prompt should be the same prompts that you would be using assuming the model is functional.