Fine-tuning with Contextual Information Beyond Prompt-Response Pairs: Possible?

Can I fine-tune an OpenAI model with data beyond just prompt-response pairs? Specifically, can I include additional information or context to influence its responses based on the given prompt?

I’m interested in making a fine-tuned model so I can quickly make high-quality flashcards that follow a very specific format, and I want to provide a predefined set of properties/questions that influence the model’s responses to a given prompt. Think of it like giving the model a “knowledge base” to draw upon.

Thank you in advance for the insight!

2 Likes

Hi!

It is not possible to teach models new knowledge via fine-tuning, so my inclination is to respond with a flat ‘no’.

However, you can definitely fine-tune a model to produce the desired output in the specific format you are looking for. The standard goal would be to reduce the prompt size by tuning the model to understand how the output should look without needing to explain it every time.

For example, let’s say we call the instructions on how the flashcards should look “additional instructions.” Fine-tuning would result in a net benefit if the cost of including these additional instructions is greater than the cost of using the fine-tuned model with only the context needed for creating the flashcards.

In general, you will only need to fine-tune if you can’t get the model to produce the desired output consistently. If you can already achieve the desired format, you might also consider using the batch API. It has a low cost but can take up to 24 hours to deliver the results.

The decision depends mainly on these factors and your specific use case.

Hope this helps.

3 Likes

Hiya,

The models already have a Knowledge Base available to them?

If you’re making a CustomGPT that’s what folks have been calling it, the “knowledge base,” but if you’re working with the API it’s File Search and then Vector Storage. I can’t tell which you’re wanting to work with, CustomGPTs or the API?

It’s not possible to “Fine Tune” a CustomGPT yet.

Fine Tuning for the API is for use cases when your System Instructions (Custom Instructions for cGPTs) are too long. The prompt-response pairs you’re talking about are uploaded via a json in a very particular format.

There isn’t another way to officially Fine Tune a model yet, but the methodology—providing detailed prompt/response examples of what you expect—works just fine. I attempt some “Visual Fine Tuning” here.

Regardless of the Interface you’re using, I think you should be able to achieve what you want through a well defined template in your Instructions that references files in your Knowledge Base wherever it happens to be. Some of those files are a well organized document that has your properties/questions. Another file provides detailed examples of the interactions you expect.

Just make sure the “interactions” are well-structured, like a screenplay and that each interaction follows the same format.

1 Like

If you use GPTs or Assistants you could accomplish this with few-many shot prompting. I believe you could get it done with 1 shot but I’d have to evaluate responses.

Give the model all of the instructions to create the flash card. Then in a text file include perfect examples. Not examples you think are okay, make sure they meet all of your expectations, the model will follow this specifically. This could be 1-infinity. 1-3 is ideal for less expensive prompts.

It helps to give input before the example so the model knows how to adapt to each input.

<example 1>
input: make a flash card about cheesecake flavored chicken nuggets
Output: [Flashcard details about a cheesecake flavored chicken nugget}
</example 1>

Then at the end of the prompt tell the model “I’ve provided perfect examples in your knowledge. Use these examples with each query”

If you need more help, let me know.

Creating datasets, paying for training, and using a less capable model for a task like this may not be the best approach.

1 Like

Hello! Thank you for the insight!

Based on my use case, what do you recommend I use? regular chat, an assistant API, a fine-tuning API, CustomGPTs, or something else? I should mention that I haven’t tried any of the following yet.

I am currently looking at the Assistant API since I only have to pay for what I use, I have more control over parameters, and I can update its knowledge base anytime compared to CustomGPTs that require Plus to update my GPT (these are only claims based on what I’ve heard from others). Can the Assistant API also be trained with prompt-response pairs?

Based on what you recommend, what prerequisite knowledge do you think I should have first? Is knowing the basics of Python and reading the documentation enough? Will I have to learn machine learning fundamentals, deep learning basics, NLP, data preparation, and model fine-tuning first? I am very open to learning about these topics since I will be covering them soon as part of my education anyway.

I’m sorry for the mountain of questions!

1 Like

Hi hi! Thanks for the info!

I was talking about the OpenAI Fine-Tuning API when I made the post.

Also, I just checked out the “Visual Fine-Tuning” post you did. The ability of AI to redraw an image and correct minor mistakes, such as the “oe” or the dragon’s body being twisted in the original logo, is really fascinating. The first photo of the bone golem was my favorite! Again, I’m very new to this!

I’m going to read more on that later.

Lastly, I know it’s a whole essay, but do you mind answering this for me? Fine-tuning with Contextual Information Beyond Prompt-Response Pairs: Possible? - #5 by kentashtley

Helllloo! Thank you so much for this!

I’m now deciding between a customGPT or an assistant API. Do you think I need specialized knowledge to make an assistant API? Or is knowing the basics of Python and reading the documentation enough?

Lastly, can the Assistant API also be trained with prompt-response pairs?

Thanks again!

Just turn data into a PDF and attach to prompt the query as you wish and by the way your chats have memory and it does learn as you feed it info that’s the training

This is incorrect.

:hugs:

Since you’re new to this, I suggest familiarizing yourself with the documentation which answers many of your questions.

You’re a good researcher. You’ll become comfortable with ‘the documentation’ when you understand it’s the program-in-question’s single source of truth.

The prerequisites you’re after are very hard to define for AI. Everyone here comes from different backgrounds. Even though there are a ton of developers, even they square off into Python, Node JS, and PHP (and other) camps. First figure out what you want to do, then follow your interests. I’m picking up Python, as it seems the preferred way of communicating.

LinkedIn Learning is the place for you. They are dropping classes on all of this stuff regularly on every topic you’ve listed. They are well structured and authoritative.

Given your lack of experience programming and verbosity, I recommend ChatGPT Plus to start off with. It is $20 per month for GPT 4o access; alternatively, 128k of Token output from GPT 4 is $15. API calls can balloon quickly if you’re not prompting efficiently—which you’re not likely to do as a beginner. Later, you can take everything you learn from experimenting with a CustomGPT when you’re ready to build an Assistant.

Meanwhile, while you do need to learn quite a bit to be able to use the API programatically, you do not need to know much programming to begin training an Assistant (which is what you will access through the API). OpenAI provides a playground that is basically a more advanced version of the ChatGPT UI which allows you to train various models without needing programming.

In my work and journey, I started with ChatGPT Plus, then quickly went to Teams because I was still running into limits. I studied all of the principles you’ve listed. Now, as I start using Assistants, I often bounce between the two Types to refine my language and prompting in the ChatGPT UI to later leverage the power of the Assistant.

1 Like

if you use the playground it should be easy to use with no-code. No need for all the extra skills. Hit me up on discord if you need help with setup its should be relatively easy. discord jerry5555

This is a high level view of how I’d set this up in playground:

The “I have included perfect examples in your file search. Use them in every query.” instruction is mandatory. It can be reworded but it’s to ensure that the model at the examples before creating a flashcard. Without this instruction the model will pick and choose, on its own, when to use the examples.

This is the examples.txt file:

Change the input to prompt/phrases that you would actually use when generating them so the model has a better understanding and can easily recreate what you need. the more examples the better, but also higher cost.

Again it is important to provide high quality examples. This is what the model will recreate.

Btw this isn’t training. I see it more as a form of incontext learning.

2 Likes