Fine-Tuning In a Nutshell with a Single Line JSONL File and n_epochs

[osi]

Sorry I have been trying to figure out how to use what you sent with the script I fine tuned with originally but do not know how to add it, I have only been coding for 2 weeks and am massively struggling. I found this code that has the epoch variables, can you explain which parts I need to change in this script to beable to run it please?

import openai
import requests

openai.api_key = “INSERT_YOUR_API_KEY_HERE”

def fine_tune_model(prompt, dataset, model_engine=”davinci”, num_epochs=3, batch_size=4):

  headers = {
   “Content-Type”: “application/json”,
   “Authorization”: f”Bearer {openai.api_key}”,
   }
  
  data = {
   “model”: f”{model_engine}-0",
   “dataset”: dataset,
   “prompt”: prompt,
   “num_epochs”: num_epochs,
   “batch_size”: batch_size
   }
  
  url = “https://api.openai.com/v1/fine-tunes"
  response = requests.post(url, headers=headers, json=data)
  
  if response.status_code != 200:
   raise ValueError(“Failed to fine-tune the model.”)
  
  # Get the ID of the fine-tuned model
   model_id = response.json()[“model_id”]
   return model_id

[ruby_coder]

Hi @osi

Please edit your post and format your code blocks using Markdown triple-backtick as follows:

```
# code here
```

Unformatted code is really hard on the eyes when reading it.

Thanks.

[ruby_coder]

@osi

data = {
“model”: f”{model_engine}-0",
“dataset”: dataset,
“prompt”: prompt,
“num_epochs”: num_epochs,
“batch_size”: batch_size
}

@osi

I don’t use Python, and so I do not know how the Python OpenAI wrapper you use works, but if you review the fine-tune directions, you have to upload a file first and pass the file id to the fine-tune create method.

Also, the OpenAI API fine-tune create endpoint parameter is “n_epochs” and not “num_epochs” etc. so you will need a Python OpenAI API wrapper person to help you because I do not use the Python OpenAI wrapper at all.

Sorry about that. I stopped programming in Python around 3 years ago in favor of Ruby, hence my username @ruby_coder . It’s a matter of personal taste and style.

In other words, I am sorry but I cannot help you debug your Python OpenAI API code.

In the Ruby gem OpenAI API wrapper, the names of the OpenAI API params and the Ruby gem OpenAPI wrapper params are 100% the same so there is no confusion.

@sps, can you help @osi ?

Thanks!

[sps]

I haven’t used python to fine-tune any models, instead I’ve been using NodeJS.

However there’s docs using which @osi can set hyper params using the openai cli tool.

openai api fine_tunes.create \
  -t file-JD89ePi5KMsB3Tayeli5ovfW \
  -m ada \
  --n_epochs 1

following this example, all @osi has to do is use -- followed by the parameter name which is followed by the value.

So n_epochs would be specified like --n_epochs followed by the value.

Hope this helps.

[renestavnes]

Thanks for the tutorial–it’s very helpful!

One question… Is the graphical user interface you use (OpenAI Lab that runs on localhost) provided by OpenAI, or is it something you’ve created yourself?

In the first case: how do I get access to it?
In the last case: how do you interact with the openai api to get the data (about your models)?

[osi]

Sorry for all the edits, something always seems to be going wrong. I keep getting an error that the stream was interrupted while fine tuning and it keeps happening even when I use a different machine. Any idea what could be causing that to happen?

Ok all good, I just had to keep running it and eventually it worked, cheers!

If anybody has the same error, uninstall the latest version of openai and install 0.22 for the fine tuning, that worked for me.

[ruby_coder]

@renestavnes

Thanks for the tutorial–it’s very helpful!

You are welcome, @renestavnes My style is to “speak” though code and real world examples and lab work; so I am seriously and painfully biased toward software developers and not a really not a personal fan of “hand waivers” giving advice or doing the “bait-and-switch” posts looking to profit off forum members.

I am here only to help people who need help from someone with many decades of experience in IT systems, network, cybersecurity and software development. The OpenAI API, while not “perfect” is fun and easy to use.

Sorry, I don’t have time to make YT videos because I code and making YT videos just take time from “the real work” which is coding; so I am happy you found my bare-metal “just code and do the work only the facts” approach useful.

@renestavnes

One question… Is the graphical user interface you use (OpenAI Lab that runs on localhost) provided by OpenAI,

No.

@renestavnes

is it something you’ve created yourself?

Yes, it is a Ruby-on-Rails application which I wrote myself, from scratch which I coded in my spare time because writing code is so much fun. Normally I code during morning coffee or when my wife is cooking dinner, haha.

@renestavnes

In the last case: how do you interact with the openai api to get the data (about your models)?

Because my lab setup is a Rails app, the OpenAI wrapper is a Ruby gem called “ruby-openai”. I will paste the link in core technologies references below. Then on top of the core “ruby-openai” wrapper, I create Ruby modules which perform all the basic API functions which I use in Rails controllers, since it is all web based.

I also use a DB to store all completions, embeddings, user data, etc. I store all fine-tunings file locally with unix timestamps and keep a running history of all fine-tunings.

For localhost DB, I use sqlite3 because it’s faster than mysql and postgresql for single-user mode and easy to use with Rails OOTB. However, sqlite3 is not great at full-text searches out-of-the-box (OOTB) so I recommend if you want to compare full-text searches to vector-based semantic searches, you consider DBs like mysql or postgresql.

Hope this helps @renestavnes

Core Tech References:

• ruby-openai

• ruby-on-rails

• Bootstrap

• JQuery Datatables

[ruby_coder]

@osi

Ok all good, I just had to keep running it and eventually it worked, cheers!

Yes, when you create a fine-tuning, OpenAI will assign you a fine-tuning ID. You can use this fine-tune ID to check the status of your fine tune so you are not wondering what is going on. It can take hours for a fine-tune to bake at times; however lately it seems faster than a week or so before. My guess is that OpenAI continues to work hard behind-the-scenes to scale up their infrastructure.

[osi]

I just finished training on 16 n_epochs and am still getting really bad responses to my prompts, no idea what I am doing wrong.

All good, the way it accepts prompts seems to be messed up but I figured it out. I just need to type only the variables as my prompt for it to make sense.

[ruby_coder]

@osi

no idea what I am doing wrong

Then you @osi should post your JSONL file data, wrapping your data in Markdown triple backticks so the data is easy to read.

Then we shall see what your actual training data looks like, how it is formatted based on OpenAI data formatting requirements, how are your separators and stops, etc.!

[renestavnes]

Thanks for a thorough reply. It seems like you’ve got a well-rounded IT background, and the playground you’ve made for yourself is impressing. Especially so if the only times you’ve reserved for hobby coding are those situations

I’ve created a few minor fine-tunes using openai’s CLI, but when it comes to interacting with the models I find the CLI limited… (or maybe just poorly documented)

I don’t have experience with the technologies you’ve used, so I’ll see if I can create a playground for myself in a language I know (and that has an api wrapper library). It’s probably gonna be Node.js–I’m actually an app developer, but an app-frontend isn’t convenient for the purpose of experimentation and configuration

[ruby_coder]

@renestavnes

CLI, but when it comes to interacting with the models I find the CLI limited… (or maybe just poorly

Hi @renestavnes

I deleted the CLI off my system right after I installed it and have no intention of ever using it.

The API is the way to go.

[kevin15]

@ruby_coder hi,May I ask what software is being used in the picture and could you share it?

[ruby_coder]

@kevin15

May I ask what software is being used in the picture

Hi @kevin15

Thanks for asking. To be honest, I answered that question a few posts above:

Fine-Tuning In a Nutshell with a Single Line JSONL File and n_epochs

You are welcome, @renestavnes My style is to “speak” though code and real world examples and lab work; so I am seriously and painfully biased toward software developers and not a really not a personal fan of “hand waivers” giving advice or doing the “bait-and-switch” posts looking to profit off forum members. I am here only to help people who need help from someone with many decades of experience in IT systems, network, cybersecurity and software development. The OpenAI API, while not “perf…

Regarding:

@kevin15

could you share it?

If you mean sharing my OpenAI lab Rails application, I’ve not been in a rush to update it lately or publish it because of the soon-to-be-released ChatGPT API; and I plan to update my lab and reenergize developing it after OpenAI releases their ChatGPT API.

Also, I don’t want to write more code (and maintain it) which will be OBE (obsoleted) by the “soon to be released” ChatGPT API.

Hope this helps.

[ruby_coder]

@renestavnes

I’ve created a few minor fine-tunes using openai’s CLI, but when it comes to interacting with the models I find the CLI limited

I installed the CLI and deleted it less than 30 minutes later, so IKWYM.

@renestavnes

but an app-frontend isn’t convenient for the purpose of experimentation and configuration

Totally understand.

I have used node.js a few times, but as mentioned after many years of coding in languages like Python and Javascript and PHP, etc. I try to only code in Ruby and Rails these days, regardless of what is trendy or not in social media. Hence my username here @ruby_coder. After I started coding in Ruby three years ago, I stopped all Python and PHP coding for new projects; but of course, it’s a matter of style and personal preference.

@renestavnes

Especially so if the only times you’ve reserved for hobby coding are those situations

I created my OpenAI API Rails “lab” app to help answer user’s questions here with “authority” based on coding every line and running every test personally (no “hand waving”).

Also, I am also the sole software developer of a huge commercial Rails back-office accounting application, which I still code commercially weekly when the ops manager asks for more features, etc. Hence, I always have VSC up on my desktop main screen and anyway, coding (in Ruby) is easy and fun; so “why not?”

HTH

[mba]

As I stated before, this tutorial is very helpful. I just fine-tuned Davinci with around 100 prompt/completion pairs from our support database. I used n_epochs=16 and can confirm that if I test my fine-tuned Davinci model by passing the exact prompt from the training data, I get the exact completion from the training data as a response. So far so good. However, what I currently do not understand is the benefit of such simple prompt/completion scenarios. A simple database would be sufficient to get the same result. So I slightly changed the prompt for the same topic (a more formal question for the same problem) and got a completion that was completely wrong. Excuse me, I’m new to this topic, but I’m really impressed by ChatGPT’s capabilities. Therefore, I’m trying hard to get a similar system working (fine-tuned with our domain data) to support our call center agents. I would be really grateful to hear your thoughts on this. Thank you.

[ruby_coder]

@mba

However, what I currently do not understand is the benefit of such simple prompt/completion scenarios.

Hi @mba

See this follow-on discussion:

Model Fitting In a Nutshell with a Single Line JSONL File and n_epochs

This tutorial / lab experiment is a continuation of an earlier tutorial, Fine-Tuning In a Nutshell with a Single Line JSONL File and n_epochs, where we focused on insuring a single-line JSONL file would get the “expected” results from the same completion prompt used in the training file. During that tutorial, our machine learning experts correctly commented that by using 16 n_epochs the tutorial (me) had inadvertantly created the ML sin of overfitting. @georgei cornered me in a good way to …

[mba]

Thank you for answering. I read your follow-up post and the discussion. So far, it is clear that if you have limited training data, you should use a higher value for n_epochs than the default value of 4. However, I am curious about the number of training data required for the model to provide useful answers, like ChatGPT does for a wide range of knowledge domains. I only need it for our domain, for which I have provided the training data

[ruby_coder]

@mba

. However, I am curious about the number of training data required for the model to provide useful answers, like ChatGPT does for a wide range of knowledge domains.

That depends on your training data, model, etc and so you will need to experiment and fine-tune for your particular data and use-case to find the correct model fitting.

There is not ‘one value’ for n_epochs which works for all, that is why it is a tunable parameter.

That is also one good reason it is called “fine-tuning”.

HTH.

[AgusPG]

In order to get meaningful (and useful) results when fine-tuning a model, it’s important to understand what’s going on under the hood. It’s true that OpenAI has democratized access to LLM to a broad community of developers, and I really celebrate this fact. But fine-tuning is not as trivial as it might seem from an “utility” perspective. At least, it’s important to understand the difference between training/validation data and the overfitting/underfitting dilemma (or the bias/variance tradeoff). I strongly recommend checking these concepts out before spending resources on fine-tuning one of these models.

What @ruby_coder has brilliantly showed in this tutorial is that these models can learn. That is: given some training data and enough training time (enough epochs), their parameters can be manipulated to reproduce patterns that occur in the training data. In fact, this is a very common technique to check out that your neural net has been properly configured when you code it from scratch and does not have catastrophic bugs: you try to overfit it to just one training sample and should get a perfect completion as a result. This is the reason why I like this tutorial.

But the generalization power of these models is a whole different aspect, and it’s what makes them powerful (as you were pointing out @mba). On this regard, OpenAI built-in tool offers a very valuable option: the validation dataset. As the training process goes on, you can check out some basic metrics in your validation dataset (that is, one that the model is not using to reconfigure its weights). If the evolution of these metrics is favorable, that means that the learning process (that uses the training data) is enhancing the generalization capabilities as well, on the in-domain validation dataset that you actually care about. Whenever you see that your validation metrics start to degrade, the learning process becomes a memorization one, that narrows the generalization power of the model (this is not what we want in 99% of the cases).

I know this all sound pretty theoretical, but I strongly believe that it’s really hard to get useful results out of fine-tuning without having some theoretical understanding of these concepts. At least, up to a certain point. Hope that helps!!