1

We have built a custom-trained model using our application-specific data. Now we wanted to utilize the model to answer questions related to our product using the custom-trained model. Any questions outside the scope of the custom-trained model need to be routed to the text-DaVinci-003 model to get an actual response. Please let us know if we can set a fallback model like this through the API call.

2

Just a fast OTTOMH rough example, @arun.raja, you can take your training data prompts and put them in an array of prompts as follow:

@array_of_prompts[
   "fine_tune_prompt", 
   "fine_tune_prompt",
   "fine_tune_prompt",
   "fine_tune_prompt",
   "fine_tune_prompt",
   "fine_tune_prompt",
   "fine_tune_prompt",
]

You might store these in a DB along with the embeddings for each prompt, from a array like this:

@array_of_prompts[
  {"fine_tune_prompt", "embedding_vector"},
 {"fine_tune_prompt", "embedding_vector"},
 {"fine_tune_prompt", "embedding_vector"},
 {"fine_tune_prompt", "embedding_vector"},
 {"fine_tune_prompt", "embedding_vector"},
 {"fine_tune_prompt", "embedding_vector"},
 {"fine_tune_prompt", "embedding_vector"},
 {"fine_tune_prompt", "embedding_vector"},
]

Then, when your chatbot user enters a prompt, you can take the embedding of that prompt and compare it to the DB of embedding vectors and if there is a match with high score, you can route the query to your fine-tuned model. If there is no match with a high score threshold (which you can get by experimentation), then you can route to another model.

The above is just an outline, you can modify this based on the programming language you are using, etc.

Hope this helps.

:slight_smile:

1 Like
4

@ruby_coder Is there a way to get embedding vector for prompts ?

5

Yes, you use the OpenAI API embeddings endpoint and send the prompt as input and you get the embedding vector back as output.

:slight_smile:

6

Do we have any alternate approach . I think this is not an efficient one since my prompts is more then 10 lakh and searching everytime is not an efficient solution