Great info. Sharing some techniques I have tried and looking for some feedback from anyone on here.
I have been playing around with a version of B in your msg. Each user prompt gets appended to the previous one in order to query for vectors and add context.
My use case is a POC for a fixed assets management app for accountants. The main question everyone has is how can we strengthen the conversation aspect.
Situation:
user: how much did the MacBook Pros cost?
assistant: $12,000.00
user: how about the Mercedes Sprinter Vans?
Solution (?):
- The text to query my pinecone vectors combines all user messages in the convo, with the most recent coming first.
So using the prior example, I query for embeddings using:
“how about the Mercedes Sprinter Vans? how much did the MacBook Pros cost?”.
This at least includes the keyword “cost” with “Mercedes Sprinter Vans”, which will help me find some info regarding the cost of Mercedes Spritner Vans. Point two is where I try to get the right context as much as possible.
- I have used a grouping methodology when creating embeddings.
For example, in our schema for a fixed asset (e.g. vehicles or computers), we have many cost related fields (purchase cost, service costs, capitalized costs, misc costs, etc.). So I programmatically create embeddings that say “Cost details for Mercedes Sprinter Van asset: Acquisition cost: $60,000, Service costs: $1,000.00, etc etc”.
This works half decently, but we have found that the key term in the second prompt (in this example, Mercedes Sprinter Van) must be very precise, and the longer the better. Say “sprinter van” instead of “Mercedes Sprinter Van” and you may not get the right context.
- I start every convo (on the backend) with a system message that tells ChatGPT to ask the user to be more explicit in their request if it doesn’t have the right context.
Ultimately this is not a 100% satisfactory approach, but the natural language capabilities of ChatGPT are so incredibly good that even when it doesn’t know the context, it makes for still a pretty nice UX.
Would love to hear thoughts from anyone who has made conversations + embeddings work nicely.