I am new to NLP and need assistance in choosing a strategy. I am currently developing a knowledge bot using the gpt-turbo model, which selects answers for user requests solely from a predetermined list of answers (answers must not be modified). These answers are specific to a particular domain, and sometimes there may be multiple correct answers. Initially, I created a few shot prompts, which worked adequately for a small list of about 300 potential answers. However, I now wish to expand the list to include thousands of potential answers.
I am unsure whether fine-tuning or embeddings with completion would be the best approach in this case. Could you please provide some guidance? There is so much confusing info on the web.
You should also consider function-calling as a technique. This would allow AI to browse a hierarchy of documents to arrive at exactly what is best instead of trying to change the weights of AI training or create embeddings semantic matches against information that is unknown to user or AI.
For my situation, I am utilizing a Python list to select answers. The user’s request may include the answer explicitly, or the model needs to be capable of generalizing and determining an answer from the list. How can function calling be applied in this scenario?
# retrieve lists of answers about our company that can fulfill user questions
functions.get_category_answers(string:enum list)
Or you can go further and have two, a search function, and a browse results function.
This would prevent continuous knowledge injection tokens or a model at 8x the price even when the user is just trying to make the bot do silly out-of-domain things.
you mean to use Search function to classify whether user request is valid. If yes, use a Browse function to output most propbable asnwer among those returned by gpt? can u please clarify
You would write a search engine that can search your documents. This can either be a normal keyword search or it can be an AI-powered vector database search.
Then you provide the AI an API function specification it can use through the model’s function-call ability.
The result provided back to the AI could be the top 5 answers, or if they are long, a document number and summary that the AI can make followup queries though a document retrieval function, to get the whole text of what seems like the best question/answer match.
This would essentially be like how ChatGPT’s “Browse with Bing” is implemented, except it is “browse with my answers”.
Embeddings is just a type of AI model that when given an input returns a vector that has semantic meaning to the number values within. It doesn’t describe any particular chatbot technique.
Vector-based semantic search is a more exact term when using a vector database to retrieve similar knowledge items.
However, a database server and a thousand embeddings is not needed.
If you augment your QA python list with handwritten metadata such as a category “login problems, billing issues, …” and provide that list of categories as a function enum (a list of items an API will accept), then the AI can retrieve all entries within a category by a single function call (without any advanced search engine). The metacode I showed quoted in my very first post.