Programmatically catch ChatGPT response when it is unable to fulfil a request?

API
1

Working with ChatGPT, is there a way to programmatically catch the cases when it is unable to fulfil a request or answer a questoin, such as ‘sorry I cannot assist you with …’ kind of answers?

My program asks ChatGPT to rewrite emails and this sometimes happens to my requests and I need to dynamically catch that and change my prompts accordingly. I don’t think this is a type of ‘error’ or ‘exception’? If not, how do you catch such cases in code?

Thank you.

2
  1. Use regex to capture any of the strings like “as an llm”, “sorry, but I can’t”, etc. etc.
  2. Use JSON mode to fill in a field, and tell it to write NULL if it cannot answer. E.g. “Return a field called ‘answer’ that will either be a string representing your answer, or NULL if you must decline to answer due to your limitations, or you are in disagreement.” (50% likelihood of working)
  3. Use a secondary call to analyze the response, and ask the LLM on a separate call, “did the LLM answer this call, or refuse to answer?”
2 Likes