Hello,
So I have a use case where I would need the gpt-3.5 to do some form of client intake. Basically, we would give list of fields to collect in a function call and then prompt it to gather that information.
Unfortunalty with 3.5 this don’t seem to work well as there might be some issues regarding hallucination (it basically work 50-60 percent of the time) espicially if the conversation gets long or other function calls are involved.
What could be the best solution to mitigate this ?
Here are a few things you can consider:
- Validate the value of the fields returned and send appropriate message back to the API.
For example, if you have a name field, it often hallucinates with variations of ‘John Doe’. So you can try to check if you get John Doe and send back { status: "Invalid name", message: "Please provide valid name", ... }.
Another example, if you have email field, if often hallucinates using ‘@example.com’ as domain. This also happens for URL field. So, just do the same as in name field, { status: "Invalid email", message: "Please provide valid email", ... }
- Check your function definition and decide which field should be required or not. Not requiring a field might prevent hallucination and you can easily trap if it comes up empty and so you can send back another status to prompt the user to input the said field.
For example, if you need name, email and phone number and you set all as required, the API often times will hallucinate and fill all fields. Or if the user only gave name and email, it will hallucinate the phone number. By not requiring the phone number, you can check if it comes empty and prompt the user to fill it.