Using Batch API for fine-tuned models

API
, ,
1

Hello,
I have a fairly large dataset, so I want to use Batch API on my fine-tuned model; how can I do this? What endpoint should I call?

I am following the tutorial on Batch API; in the examples, it uses gpt-3.5-turbo-0125 as the model. How can I change it to my fine-tuned-model ID instead?

Example of the request to the API:

{"custom_id": "request-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "gpt-3.5-turbo-0125", "messages": [{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Hello world!"}],"max_tokens": 1000}}

2

All you should need to do is specify the model name, which will start with something like “ft:gpt-4o-mini-2024-07-18:yourorg:8u33344..”

It is run against the chat completions endpoint, the same as normal inference with your fine-tuning AI model.

gpt-4o-mini-2024-07-18, gpt-4o-2024-08-06, and gpt-3.5-turbo based models have a batch discount.

2 Likes