Hello everyone! I’m curious if anyone here has experimented with integrating a fine-tuned model into the Assistant API. Specifically, I’m wondering about the challenges and benefits you’ve encountered in tailoring the Assistant’s responses using a fine-tuned model. Any insights or experiences in this area would be greatly appreciated. Thank you!
3 Likes
My understanding was that the assistants API removes most of the need for fine tuning by allowing you to inject it with specific knowledge and behaviour.
From what I’m aware of, the largest benefit of fine tuning over prompting is possibly saving money if you go through a massive amount of prompts and tokens.
I may be wrong though, if there are other benefits I’d be interested to hear them.
Hi,
Have you tried replacing the model name with your fine-tuned model in the creation object?
const assistant = await openai.beta.assistants.create({
name: "Math Tutor",
instructions: "You are a personal math tutor. Write and run code to answer math questions.",
tools: [{ type: "code_interpreter" }],
model: "your_model_name"
});
The documentation states:
- Model: you can specify any GPT-3.5 or GPT-4 models, including fine-tuned models. The Retrieval tool requires
gpt-3.5-turbo-1106andgpt-4-1106-previewmodels.
1 Like
but i think it will not have the tool retrieval.
do you think that using the retrieval, upload knowledge based documents or fine-tuning documents (in CSV, or PDF). will train the model or fine-tune it? or no?
If your requirement is for the AI to process your data and use that as context, then you should use retrievals as describe din the documentation, note that you can use the Playground uploader to get files into your assistant if you do not wish to do it programmatically.
that’s right. but it doesn’t give you the option to fine tune it anymore.
so the real question is. does the data in the documentation uploaded from the playground fine tuning the gpt-4-1106 model? or no? its just using it as a context and no training is happening?
No, uploading data to an assistant does not fine tune the model, that is a separate endpoint and workflow.
i see. so right now, there’s no way to use fine tune model + retrieval tool?
Correct, the documentation states
@nikkdev did this work for you? Were you able to use a fine-tuned GPT-3.5 turbo 1106 + retrieval tool with the Assistants API? Many thx.
haven’t tried it but i think no. we can’t use specific fine tuned model for assistant API
Anyone knows if we’ll be able to use retrieval tool on our fine-tuned models in the future?