I couldn't get expected result from sample fine tuning(web api)

API
1

I am trying to follow sample case studies but they are not working as expected for me.

I tried at least 50 times, It felt like it is time to ask for help :sweat_smile:

(the description suggests using logprobs=2 but the sample request does not have logprobs in the body, I have tried both of them but no luck.)

I used the sample dataset

{"prompt":"Overjoyed with the new iPhone! ->", "completion":" positive"}
{"prompt":"@lakers disappoint for a third straight night https://t.co/38EFe43 ->", "completion":" negative"}

Request: POST /v1/fine-tunes

{
  "training_file": "file-uWRrnta0EjZaSMwV0q3eLaRs",
  "model": "ada"
}

Response from : GET /v1/fine-tunes/ft-S3L8VL2vLSrVFADavRslRVmT

{
  "id": "ft-S3L8VL2vLSrVFADavRslRVmT",
  "object": "fine-tune",
  "created_at": 1640729340,
  "events": [...
    {
      "object": "fine-tune-event",
      "created_at": 1640729400,
      "level": "info",
      "message": "Uploaded result file: file-65pZwMUTMlmKg1yyv2yyphz0"
    },
    {
      "object": "fine-tune-event",
      "created_at": 1640729400,
      "level": "info",
      "message": "Fine-tune succeeded"
    }
  ],
  "fine_tuned_model": "ada:ft-betalgo-up-ltd-2021-12-28-22-09-55",
  "hyperparams": {
    "batch_size": 1,
    "learning_rate_multiplier": 0.05,
    "n_epochs": 4,
    "prompt_loss_weight": 0.1
  },
  "model": "ada",
  "organization_id": "org-dQWHBftxPqygzZwQ3Z6M4ixI",
  "result_files": [
    {
      "id": "file-65pZwMUTMlmKg1yyv2yyphz0",
      "object": "file",
      "bytes": 413,
      "created_at": 1640729398,
      "filename": "compiled_results.csv",
      "purpose": "fine-tune-results",
      "status": "processed",
      "status_details": null
    }
  ],
  "status": "succeeded",
  "training_files": [
    {
      "id": "file-uWRrnta0EjZaSMwV0q3eLaRs",
      "object": "file",
      "bytes": 182,
      "created_at": 1640729339,
      "filename": "FineTuningSample1.jsonl",
      "purpose": "fine-tune",
      "status": "processed",
      "status_details": null
    }
  ],
  "updated_at": 1640729401,
  "validation_files": []
}

Request: POST /v1/completions

{
  "prompt": "https://t.co/f93xEd2 Excited to share my latest blog post! ->",
  "max_tokens": 1,
  "model": "ada:ft-betalgo-up-ltd-2021-12-28-22-09-55",
  "logprobs": 2
}

Response:

{
  "id": "cmpl-4KExVHDl19jzUm6Qga6XlsRIqmD21",
  "object": "text_completion",
  "created": 1640729409,
  "model": "ada:ft-betalgo-up-ltd-2021-12-28-22-09-55",
  "choices": [
    {
      "text": " https",
      "index": 0,
      "logprobs": {
        "tokens": [
          " https"
        ],
        "token_logprobs": [
          -0.42703274
        ],
        "top_logprobs": [
          {
            " http": -1.6594573,
            " https": -0.42703274
          }
        ],
        "text_offset": [
          61
        ]
      },
      "finish_reason": "length"
    }
  ]
}
2

Hi
Not sure if this is the right answer but, from what I understood so far, in order to use fine-tunes you should provide a dataset of at least 100 samples.

4 Likes
3

hi, I was expecting the sample datasets to be working. I will add more data and test it again. Thanks for the reply.

4

Thanks, @nunodonato . I have tried with more data and I could get expected responses. I think it should be more clear on documentation that sample data is not enough.

2 Likes