Large context document and finetuning

API
,
1

Hey there,

We have a case we want to finetune a gpt-4o or mini with a considerable dataset (~500 rows) that’s aiming to give users ability to generate a (figma) design as json object. Json object is designed by our team and might contain sub-objects that just our plugin knows to read.
All design construction (json product) is based on a guide, for the sake of easiness let’s call it manifesto (or context) which talks about structure, components relation and what component could be parent, what could be child, and on, this document is around 20 pages long.

So model needs to be finetuned to answer to a user prompt on generating a web screen with our own json structure.

Do we have to send on every dataset record this long document, together with question and answer?

Or do we have just to mention and then send it as context when we do api call? We followed this practice but we are hitting max_tokens (4096) and we don’t get a full generation for a screen. Or there’s a solution for this, which we don’t know?

I’d really like to hear your guidance on how to approach this problem.

Thanks in advance!

2

I am not really sure that I completely understood the context. Here’s what I got out of the post above:

  • Your final objective is to get a custom json object as a figma design; so that you can render off of it.
  • You would get to that figma design on an interative basis through user interactions.
  • you have guidelines (aka context) that the json object representing the figma design must follow.

Those guidelines contains instructions of how components must be structured within the json object (such as where a component can appear, parent-child relationship and cross reference between components).

If my above understanding is correcting, can you please describe the function of fine-tuning and what steps you have taken to come to the conclusion that fine-tuning is the best option.

It seems that you can develop this figma design by introducing one component at a time and injecting the necessary knowledge into that one component; analogous to developing outline of a story plot, outline-of-story chapters and then building out chapters. Similar to something like this here for illustrative purpose (https://youtu.be/KJ-J5AYGG7g)