No way to delete fine tune job objects...no setting metadata, undocumented methods

API
1

I really don’t need to see failed fine-tune and fine-tuning from 2023.

Can I delete them? Seems not. Why not?

My doing many replications of issues (such as recently seeing ft:gpt-4.1-mini now failing on images), generates unwanted listings and logs, even if I was clever enough to completely validate JSONL files.

I want them deleted. Probing for undocumented methods discovers no hidden API methods.

Error deleting fine-tuning job status:405 {
  "error": {
    "message": "Invalid method for URL (DELETE /v1/fine_tuning/jobs/ftjob-DUR...)",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}

Error deleting fine-tuning job status:404 {
  "error": {
    "message": "Invalid URL (POST /v1/fine_tuning/jobs/ftjob-DUR.../delete)",
    "type": "invalid_request_error",
    "param": null,
    "code": null
  }
}

Metadata useful to stop the listings?

There is metadata at create time, and filtering based on metadata, but there is no method to modify a fine tuning job, to add metadata for filtering. You can’t add this after the fact. Further lack of possible utility, that might look like:

def mod_ft_job(job_id):
    '''add metadata - if such method existed'''
    job_id_url = f'https://api.openai.com/v1/fine_tuning/jobs/{job_id}'
    body = {"metadata": {"failed":"True"}}
    response = httpx.post(job_id_url, headers=HEADERS, json=body)
    if response.status_code != 200:
        print(f"Error modding fine-tuning job:"
              f"{response.status_code} {response.text}")
    else:
        ft_object = response.json()

Finally: the fine-tuning job object of the API reference does not show the metadata field. It IS returned by a GET v1/fine_tuning/jobs/{job_id}

Platform site inspection documents a useful query string

A query string NOT in API reference

What does work to filter jobs by status:

GET https://api.openai.com/v1/fine_tuning/jobs?limit=10&status=failed

or

GET https://api.openai.com/v1/fine_tuning/jobs?limit=10&status=succeeded

Some code to actually choose between only succeeded and only failed fine tuning jobs and return them.

import os, httpx
HEADERS = {
    "Authorization": f"Bearer {os.getenv('OPENAI_API_KEY')}"
}
def get_ft_jobs(limit: int = 20, *, failed: bool = False) -> list[dict]:
    """Return done fine-tuning jobs, with switch for succeeded or failed.
    Status actually can be:
        'validating_files', 'queued', 'running', 'succeeded', 'failed',
        'cancelled', 'pausing' or 'paused'
    """
    status = "failed" if failed else "succeeded"
    url = (
        "https://api.openai.com/v1/fine_tuning/jobs"
        f"?limit={limit}&status={status}"
    )
    try:
        response = httpx.get(url, headers=HEADERS, timeout=10.0)
        response.raise_for_status()
    except httpx.HTTPStatusError as e:
        raise RuntimeError(
            f"OpenAI returned {e.response.status_code}: {e.response.text}"
        ) from e
    except httpx.RequestError as exc:
        raise ConnectionError(f"Request to {e.request.url!s} failed: {e}") from exc
    ft_objects = response.json()
    ft_object_list = ft_objects.get('data')
    if ft_object_list is None:
        raise ValueError("Response JSON missing 'data' key")
    import json
    print(f"Got jobs; sample:\n{json.dumps(ft_object_list[:2], indent=2)}")
    return ft_object_list

# usage
joblist = get_ft_jobs(failed=True)

That still does not fulfill the need “ignore ONLY fails”.

Actions:

  • update documentation with current object returned
  • update documentation with all query strings supported
  • consider the deletion of jobs.
  • consider the addition of metadata to jobs

Also for documentation updates:

Create checkpoint permissions API reference example

The call that is in the code example does not actually show a checkpoint-style model name

1 Like