Learning Rate Multiplier Values

API
1

I’m looking for documentation on values / ranges for the learning_rate_multiplier hyperparameter in the fine-tuning API, but I’m pretty sure there aren’t actual values listed…anywhere in the docs? It says that it’s a scaling factor, so am I safe to assume that the the ‘auto’ value specified in the docs is 1? And then if I want to scale up or down, its just a fraction or multiple of that?

Thanks

2

You are not “safe” as it is not 1 by default. You might get 2 or 0.2 depending on the model, training file, or other hyperparameters specified.

1 Like
3

Ok, but where do I actually see what it was (or will be) for a particular fine tuning job? It’s not available in the response object, in the fine tuning interface, or anywhere that I can find.

Hopefully I’m just completely missing something but I can’t figure out how to find it.

4

A little python script for you:

import openai, json; c = openai.Client()
print(json.dumps(c.fine_tuning.jobs.list().model_dump()['data'], indent=1))

Result, list of jobs:

[
 {
  "id": "ftjob-1234uZHb7VWRPmJiJA43T8",
  "created_at": 1699475555,
  "error": {
   "code": null,
   "message": null,
   "param": null,
   "error": null
  },
  "fine_tuned_model": "ft:gpt-3.5-turbo-1106:orgname::12345678",
  "finished_at": 1699478999,
  "hyperparameters": {
   "n_epochs": 9,
   "batch_size": 1,
   "learning_rate_multiplier": 2
  },
  "model": "gpt-3.5-turbo-1106",
  "object": "fine_tuning.job",
  "organization_id": "org-1234uZHb7VWRPmJiJA43TeT8",
  "result_files": [
   "file-1234uZHb7VWRPmJiJA43TeT8"
  ],
  "status": "succeeded",
  "trained_tokens": 97769,
  "training_file": "file-1234uZHb7VWRPmJiJA43TeT8y",
  "validation_file": "file-1234uZHb7VWRPmJiJA43TeT8z",
  "user_provided_suffix": null
 },
 {
  "id": "ftjob-1234uZHb7VWRPmJiJA43TeT",
  "created_at": 1699475011,
  ...
1 Like