How do I determine whether a question will be answered by embedding or by chatgpt?

I have customized my model to respond the way I want, using my data on certain topics, taking advantage of embeddings. However, it should not use embeddings when answering all questions, in some cases chatgpt should also use self-generated answers. How can I determine whether to use embedding or chatgpt when creating an answer to a question.

1 Like

You could have a classfier before the GPT generation calls which classifies the question into whether it is for data for the embeddings or the chatgpt one. This would allow you to sort of use if else to make the correct gpt call to use

1 Like

Thank you very much, what you mean is to take advantage of an ml classification model. Is there a document, cookbook, or resource that will guide me to do this ?

You can find plenty of resources online that can help with question classification.

Another possible solution could be to prompt GPT as well by giving it the topics which you are using the embeddings for and asking whether the question will be answer by these topics or not, though this method will be very temperamental and you will need to structure the prompt quite well.

I really couldn’t find a proper document that would work for me, and I didn’t think about how I could make the classifier according to what. Can you send me a document as a suggestion?

same, I’m looking for something similar but no dice

There’s classification use cases in OpenAIs documentation under Fine Tuning.

Although it could not work correctly. A model such as Ada or Babbage is trained on less data than DaVinci & cGPT (I’m assuming). Probably would result in false positives

I may be wrong, but I always include embedding contex. I wouldn’t embed information that a model like cGPT already knows

i’m using the gpt-3.5-turbo model and unfortunately there are no fine-tune on this model. I’m obligatorily using embeddings to generate an answer with my data. My main problem is that when a question comes, how can I distinguish whether it is based on embeddings or not, the answer will be generated in the normal way ?

You should already know if the questions need embeddings or not as they are supplementing the knowledge of your chatbot. Most people use a semantic search using ada-embeddings to determine what pieces of information are relevant.

You would usually fine-tune a much smaller, cheaper model such as ada or babbage for any sort of classification task. In your situation though it’s not really necessary.

You should already know if the questions need embeddings or not as they are supplementing the knowledge of your chatbot.

How can I know exactly that. There may be a lot of type questions, there are millions of possibilities, I can distinguish whether there is a need to embed the incoming question as a person, but how do I make the model do this?

It all depends on the niche of your chatbot. cGPT does a pretty good job of having all-around knowledge and deducing from it.

The process of determining which embedding to use is automatically done through many ways - usually semantic search as shown in the article I’ve posted. The process of creating the embeddings is done by your own testing and knowledge base creation

If your chatbot, for example, is meant to be a mechanic’s aid, you’d want to prepare embeddings on specific mechanical information that cGPT fails to answer through your own personal testing. You wouldn’t bother to embed the complete series of Seinfield. You wouldn’t also embed every single socket size that exists, as cGPT already knows this information.

Maybe (context being build by semantic search):

Answer the question based on the context below, and if the question can't be answered based on the context, say "I don't know"\n\nContext: ${context}\n\n---\n\nQuestion: ${question}\nAnswer:

And if ChatGPT answers “I don’t know” you can use it without context

That’s what I do, but there are a variety of “I don’t know” responses from GPT-3.5. It usually starts with “The context does not provide” but I don’t think that’s completely reliable. It’d be nice if there were a flag included in ChatCompletionResult that indicated whether the completion could be supplied based on the given context.