Hello , currently i’m saving chunk of data inside of a vector database more specifically pine-cone, when im trying to ask OpenAI questions of this chunk of data it answers correctly but sometimes not , my questions is how to increase the accuracy of the replies and also restrict OpenAi to only answer from that database.
Hi and welcome to the Community!
As for the restriction part, including in your system or user prompt instructions along the lines of “Strictly only rely on the information provided in responding to the question. If the information do not contain the answer to the question, respond with ‘Apologies but I do not have sufficient information to provide an answer to this question’” often are sufficient to avoid the model from relying on any other information.
As for the accuracy of the replies, the changes depend on what is causing the inaccuracies. Is it because the model fails to properly take into account the information in formulating the response or is it due to the incorrect information being retrieved during the vector search?
1 Like
Is it because the model fails to properly take into account the information in formulating the response. it usually make up an answer or reply with i dont have an answer to this question
Are you able to share the prompt you are currently using? That would help to identify what’s causing the issue.
template = f"I want you to act as a Assistant that I am having a conversation with. Your name is Swift. Your task is to extract detailed answers from the database promptly, including all minor information and personnel details without seeking additional context. Respond comprehensively and affirmatively. Failure to do so will result in penalties. Answer a question given in a natural,human-like manner. If the answer is not included or If you don’t know the answer, just say that you don’t know. Don’t try to make up an answer."
Ok, for now I am just focusing on the incorporation of the information.
There are two observations regarding your current prompt:
-
It is not clear yet where you include the information that is retrieved via the vector search. The retrieved information should be incorporated into the prompt, i.e. you should include a variable for it.
-
In your wording of the prompt you could be even more explicit in referencing the information the Assistant should consider in the response.
Taking both into account, a possible revised version of your prompt could be as follows:
f"You are a conversational AI assistant by the name of Swift. Your task is to respond to user questions in a natural, human-like manner. In generating the response you rely exclusively on the information retrieved from the database provided herein without seeking any additional context. Your response must be comprehensive, taking into account all the specific details contained in the information from the database such as personnel details. If the information from the database does not contain the answer to the question, you respond with ‘Sorry, I don’t know the answer’. Information from database: {retrieved_information}"