Hello everyone! I’m curious if anyone here has experimented with integrating a fine-tuned model into the Assistant API. Specifically, I’m wondering about the challenges and benefits you’ve encountered in tailoring the Assistant’s responses using a fine-tuned model. Any insights or experiences in this area would be greatly appreciated. Thank you!
6 Likes
My understanding was that the assistants API removes most of the need for fine tuning by allowing you to inject it with specific knowledge and behaviour.
From what I’m aware of, the largest benefit of fine tuning over prompting is possibly saving money if you go through a massive amount of prompts and tokens.
I may be wrong though, if there are other benefits I’d be interested to hear them.
Hi,
Have you tried replacing the model name with your fine-tuned model in the creation object?
const assistant = await openai.beta.assistants.create({
name: "Math Tutor",
instructions: "You are a personal math tutor. Write and run code to answer math questions.",
tools: [{ type: "code_interpreter" }],
model: "your_model_name"
});
The documentation states:
- Model: you can specify any GPT-3.5 or GPT-4 models, including fine-tuned models. The Retrieval tool requires
gpt-3.5-turbo-1106andgpt-4-1106-previewmodels.
2 Likes
but i think it will not have the tool retrieval.
do you think that using the retrieval, upload knowledge based documents or fine-tuning documents (in CSV, or PDF). will train the model or fine-tune it? or no?
If your requirement is for the AI to process your data and use that as context, then you should use retrievals as describe din the documentation, note that you can use the Playground uploader to get files into your assistant if you do not wish to do it programmatically.
that’s right. but it doesn’t give you the option to fine tune it anymore.
so the real question is. does the data in the documentation uploaded from the playground fine tuning the gpt-4-1106 model? or no? its just using it as a context and no training is happening?
No, uploading data to an assistant does not fine tune the model, that is a separate endpoint and workflow.
i see. so right now, there’s no way to use fine tune model + retrieval tool?
1 Like
Correct, the documentation states
1 Like
@nikkdev did this work for you? Were you able to use a fine-tuned GPT-3.5 turbo 1106 + retrieval tool with the Assistants API? Many thx.
1 Like
haven’t tried it but i think no. we can’t use specific fine tuned model for assistant API
1 Like
Anyone knows if we’ll be able to use retrieval tool on our fine-tuned models in the future?
2 Likes
Has anybody actually managed to get the Assistant API to work when using a fine-tuned model? I know the docs say this is possible, but I can’t get it to work in practice. I create a new assistant using “gpt-3.5-turbo-1106”, it works, but if I specify a fine-tuned model (with gpt-3.5-turbo-1106 as a base) instead, I get an error when trying to create the assistant. FWIW, I am not using the retrieval tool, only function calling.
1 Like
I need to be able to combine those 2 as well. Hope it gets implemented soon.
1 Like
Is there any update on this? Would be nice to have the ability to use a fine tuned model with the assistant so you can provide less instruction and more examples of what an ideal output would look like.
1 Like
+1
This feature would be super useful.
@dbenito I know the docs say this is possible, but I can’t get it to work in practice
That’s been my experience, as well.
I’ve found the best solution is fine-tuning only with limited-length text files. Anything else and I’m unable to get it to work.
Start with fine-tuning a model with a single text file, just to confirm you’re doing everything correctly. Once you’ve done that, you can continue to move forward with a better idea of where it breaks.
I hope that can help 
I create a new assistant using “gpt-3.5-turbo-1106”, it works, but if I specify a fine-tuned model (with gpt-3.5-turbo-1106 as a base) instead, I get an error when trying to create the assistant.
Any progress on that? I see the Assistant setup from Playground is not allowing the selection of my fine-tuned models.
I wonder if the Assistant API allows for that.
The Assistants API does not currently allow fine-tuned models to be used, it was in the documentation for a brief period, but that was in error. Hopefully this becomes a feature at some point in the future.
2 Likes