Is it possible to configure a fine tuned model, such as GPT 3.5, with file retrieval capabilities? I cant see this option available in the UI, nor in the API.
The idea is to have a fine tuned model able to retrieve data from documents, just like an Assistant is able to do.
You can put a gpt-3.5 model in assistants and use it with file search now.
It is a “tool”, and you can’t place tools yourself in bare form and make them functional alone.
You can replicate file search or the way retrieval worked with functions, which the AI can be trained on. The AI already knows how to use functions by the spec you must supply anyway. The refinement you would do would have to be particularly useful, like training to produce HyDE into the function without needing to describe extensively how to make the provisional answer.
Function attempts at fine-tuning have ended up with many borked models, though.
That’s awesome @_j. I didn’t realise you can select a fine-tuned model when building an Assistant. Therefore, in theory, we can use the same code, functions and file search as before with an Assistant, having it use a fine-tuned model.
Thanks a million!
Hey
That’s quite a cool approach. I have also been asking myself the same question. Can you elaborate on what you mean by “You can replicate file search or the way retrieval worked with functions, which the AI can be trained on.” I’m using the native File Search from the assistants API and not some other RAG solution. With File Search you unfortunately can’t see see exactly where and how the search query is being generated and where exactly the models retrieves the search results. In my mind, this makes it rather hard to fine tune for File Search.
How exactly would you go about it?
The AI is provided “# Tools”, special ways that it can emit an output starting with special tokens that trigger the API backend to not send YOU the output, but to send to OpenAI’s internal tool recipient, to turn off logprobs, to inject special instructions only when the AI is in the tool response context, etc.
Under that hierarchy of tools are internal tools like code interpreter, the assistants file search, and ones that openai keeps for themselves, like browser, dalle, or bio (memory), or even a guardian tool to try to hide from users instructions to the AI to deny voting information (before they trained models to simply produce refusals). “Functions” is just a single tool like these others.
You cannot truly create your own “tool” as OpenAI denies you this level, and also blocks exposing special tokens in output and within logprobs to analyze the quality of tool calling.
So, whereas assistants file search has one method as a tool - send a search query to a document store where the AI has no idea the scope of what it will find to get back results without relevance, to replicate this, you would have to offer your own search code as a function to begin the process of leaving assistants and their poor everything behind for your own knowledge search the chat AI performs (which is not the best way, either).
Or just leave OpenAI completely to a platform that is not adversarial at its core. Leaving OpenAI is the cool thing for even their own employees.
Thanks!
Yeah I get that. I guess I misinterpreted your original comment. Using a different RAG Tool and using it with function calls, which can be fine tuned, was also a possibility I considered. However, given how easy their “out of the box” solution was to implement we opted for that first to do some initial testing. It’s a pity they don’t yet allow fine tuning for the file search tool like they do with function calling, or offer a way to get any insights to how it works for that matter. For now, their basic file search covers most of our requirements, but we might jump ship if the requirements change and they don’t expand their service in the near future, like they said they would in the documentation but I gues “in the coming months” is a vague enough term for them to hold off on actually deliver something in a reasonable time frame.
1 Like