I am making a purchase assistant chatbot. The idea is to have a workflow where the user can create an order via a chat interface. However, the problem is that at every step, I need to call an API.
- Select an item → Call an API to validate the item and retrieve internal IDs.
- Select quantity → Call an API to find stores where the item is available in that quantity.
- Ask the user to confirm the order → Call an API to create an order.
The problem is that there could be many different cases where the message is not valid or is only partially valid. For example:
- The item is not available.
- The item is not available in the requested quantity.
- The item will only be available in three days.
- The user types several items and several stores they want to buy from, meaning I need to call multiple APIs to validate everything and possibly create two orders.
I first tried using function calling, but it didn’t work well. Pure prompting did better. I return JSON with different fields that help determine which API should be called at each step and with which parameters. Large examples helped a lot, I believe. However, I still experience hallucinations, where I sometimes get different responses to the same question.
I am using GPT-4o-mini at the moment. Would a different model give better results?
Any advice on how to improve this? Pure prompting with many examples seems to be working, but I’m not sure if it’s the right approach.
P.S. Fine tuning is on the roadmap and vector knowledge base is connected.