What is it about Assistants and Fine-tuning?

Documentation
1

What is the difference between Assistants and Fine-tuning? And can I train the Assistants model first and then use Fine-tuning?

1 Like
2

Assistants is a high-level agent framework API by OpenAI. Basically adds some functionality and behavior on top of the OpenAI API.

Fine-tuning is the process of giving a model additional training related to how it should behave (but not what it needs to know).

2 Likes
3

Well, okay, I got the documentation right. But another question is, can I use trained “Assistants”, which I just tested at first and realized that at some points the bot does not respond the way I need, I can use “Fine-tuning” to train the bot?

4
  1. You need to clarify what exactly you mean by “trained assistants”, because there is no such generally used terminology (at least yet).
  2. Assistants in general are in beta and it is hard to have them answer the way you want.
  3. You can fine-tune a model, not the bot.

P.S. Man, you freaked me out!b When I got a notification about your answer with quote in russian, I thought I started to speak russian in the forum :slight_smile:

1 Like
5

Oh. I used the translator on the page, and forgot to turn it off. That was funny. :upside_down_face:

2 Likes
6

Transmit your data in such a way that the model is based on the information I provided when responding

7

I understood everything. Thanks a lot for the reply

1 Like
8

Happy it helped :slight_smile: Good like on your jouney!

9

Written under 20 hours ago…

2 Likes
10

Hey, I was wondering if you could share what your solution was. Im looking to train a gpt model on industry specific data and get specific behaviour on replies.
I was thinking about it the same way as you, first train the assistants on the data then use fine tuning to get the replies I want.

11

Does the assistants feature use both fine-tuning and RAG under the hood, intelligently as appropriate? So basically does it make it less necessary to implement fine-tuning and RAG from scratch?

12

I am new in this domain. Fine tuning a model and adding instructions to model seems very similar to me. In my use case I want to get the model trained on my database schema, so that it understands the tables and its columns properly to generate queries. Assistant works well when I provide the schema in instructions but would it be good if I am fine tuning the model with schema metadata? Any suggestions? Which one would be cost effective?

13

First: are you actually inquiring about Assistants, and is that the pattern you want to be using?

Assistants is best if you want server-side retention of a chat history, and plan to have users engage in a continuous chat context. Alternately, it can be because you want to use OpenAI’s file search or code interpreter instead of implementing your own solution for knowledge retrieval or running code by function calling.

That you are providing a schema in instructions indicates to me that you are not having a continuous chat, where the AI might invoke a function to call upon your database in order to answer, but rather that you are giving a single input, and expecting the database query to be the output. Is that correct?

If so, Chat Completions is the better endpoint for such an AI task that is not generating a friendly response to a user.

Then for fine-tuning - it has an upfront investment of time and money, as you must develop a significant training set of examples that mimic actual use that would inspire correct output, while not damaging the underlying qualities of the model to follow instructions. Training is by example, not by instruction, although they can be partnered.

One goal would be to reduce the reliance on instruction-following and the need for AI’s ability to follow instructions given to it. You don’t have to explain a schema with more input tokens if the AI model is predisposed through training to accept a user input “how many new sign-ups this week”, and to then produce an output that is not an attempt to answer the question (default behavior), but is a formatted output with actionable database query in the correct form.

The first step would be to fine-tune on the exact same system prompt, user input, and desired output with many varieties of all anticipated inputs, including all the instructions already necessary for success, and use a low number of epochs as hyperparameter.

See if you are able to improve the quality of what the AI writes with that fine-tuning, to know if it is worthwhile to go to the next level of fine-tuning, which is to start to remove some of the explicit instructions, because the operations have been well demonstrated in training examples and the AI should infer how to make similar outputs.

Cost tradeoff is: train for $20 or $200 of training tokens of examples - then for how much savings in input that you don’t have to provide for every call in usage. Any how many experiments are failures…

14

My usecase workflow consists of the following steps:

  1. User Query Input:
    Users provide a natural language query, such as “How many new sign-ups this week?”.
  2. Natural Language to SQL Conversion:
    The input query is processed using a Large Language Model (LLM), which translates it into a SQL query. The LLM is fine-tuned or configured to understand the database schema to generate accurate SQL queries. I want to understand how should I be implementing this. One way is to give instructions which describes my schema and all tables with table columns to assistant and assistant replies back with SQL query, other is by fine tuning. Have implemented assistant and it seems that it is giving good results.
  3. Database Query Execution:
    The generated SQL query is executed against the database to retrieve the relevant results.
  4. Human-Readable Answer Generation:
    The database results, along with the initial user query, are fed back into the LLM to generate a concise and human-readable response.
    Example response:
    “There were 120 new sign-ups this week.”
15

It seems you would want to have a conversational chatbot that can use function-calling to obtain information.

It can be left to chat proficiently, while the external AI is a specialist in writing SQL from the function call.

{
  "name": "send_query",
  "description": "Send data to an external AI-powered API for SQL queries on a company database",
  "strict": true,
  "parameters": {
    "type": "object",
    "required": [
      "user_question",
      "desired_output",
      "user_data_provided"
    ],
    "properties": {
      "user_question": {
        "type": "string",
        "description": "The SQL query or question that the user wants to ask about the database"
      },
      "desired_output": {
        "type": "string",
        "description": "The results that are expected or which must be achieved"
      },
      "user_data_provided": {
        "type": "string",
        "description": "Serialized or natural language user data that may be used in the SQL query (cross-reference with actual parameters)"
      }
    },
    "additionalProperties": false
  }
}

Chat completions is the place for an input-output scenario, with no need to make multiple API calls to set up a thread conversation and poll to await a response.

But then - do you really need an AI that can reveal unexpected patterns in data or drop tables. Perhaps you can consider individual functions, even a dozen, that are mapped to SQL queries and their results. No second AI needed.

1 Like