How can I use Embeddings with Chat GPT 3-5 Turbo

Kindly refer to the above. The strategy for making gpt-3.5 respond from a knowledge-base that we provide, seems to involve passing a “context”. We get this context by first passing the user’s question to the Embedding API and getting a response. This is where we pass a vector and get a vector back, which we convert to text. Say, the response is 3 lines of text, which represents the most relevant context for the question asked.

Now, we pass this context, along-with the user’s question to gpt-3.5, so that it generates a human like response from within that context.

The problem is that, if the user’s question is “Is it available in blue?”, if we pass it to the Embedding API, it may generate no context or multiple contexts. This is because of the pronoun used.

Hope I was able to explain myself clearly

Yes, so as I understand you, you do not pass any vectors to the model. You only send text to the model, which is what I expected.

Please correct me if I am wrong.


Yes, correct. To clarify, I just learnt about this approach while browsing this thread today and realized that there might be a conceptual problem. I have not tried this out yet.

I had the same / similar question. I think this largely will depend on the specific implementation / desired outcome, but there is a question of when to add context from embeddings when there is a conversational aspect. For instance, with davinci (single shot), its more obvious: you simply provide the context with any question and get your answer. When we have a conversational paradigm, it becomes a little trickier. We can use the previously mentioned example:

User: “How much is this shoe?”
GPT: “The shoe is $5”
User: “Is it in blue?”

What would we send to the turbo model for this kind of conversation? I see a few options:

a) Perform an embeddings search for each user query separately and add it with context to the messages array, eg:

user: Please answer the following question with context: {Context 1}.
user: How much is this shoe?
assistant: The shoe is $5.
user: Please answer the following question with context: {Context 2}.
user: Is it in blue?

This approach is problematic because “is it in blue” is unlikely to return accurate semantic matches for the query alone.

b) Perform an embeddings search for all user queries combined (eg “How much is this shoe? Is it in blue?”) and include that context, eg:

user: Please answer the question using the following context: {Total Context}
user: How much is this shoe?
assistant: The shoe is $5.
user: Is it in blue?

c) Pre-process the user queries by prompting GPT Completion to summarize, eg:

Please summarize the following user queries and determine the ultimate question: "How much is this shoe? Is it in blue?"

And get such a response: How much is this shoe and is it in blue?. This can then be searched to determine embeddings context, and we can use the rest of the approach in b) to construct the final completions prompt.

So far I’ve had some success with the “B” approach. However, more complications arise with this approach if the user decides to switch contexts and talk about something else, eg. “What about this shirt?” In that case, the embedding query: “How much is this shoe? Is it in blue? What about this shirt?” may have more difficulty finding relevant documents for context. Using distance heuristics may be appropriate to see if the the embeddings result is “close” enough.

1 Like

As per this documentation from Azure, they have suggested to pass the context as a part of System message:

Assistant is an intelligent chatbot designed to help users answer technical questions about Azure OpenAI Serivce. Only answer questions using the context below and if you're not sure of an answer, you can say "I don't know".

- Azure OpenAI Service provides REST API access to OpenAI's powerful language models including the GPT-3, Codex and Embeddings model series.
- Azure OpenAI Service gives customers advanced language AI with OpenAI GPT-3, Codex, and DALL-E models with the security and enterprise promise of Azure. Azure OpenAI co-develops the APIs with OpenAI, ensuring compatibility and a smooth transition from one to the other.
- At Microsoft, we're committed to the advancement of AI driven by principles that put people first. Microsoft has made significant investments to help guard against abuse and unintended harm, which includes requiring applicants to show well-defined use cases, incorporating Microsoft’s principles for responsible AI use
What is Azure OpenAI Service?

I have tried to pass the context related to the latest question as the system message and the I have received satisfactory answer.