Hi,
We’ve gotten a ChatGPT app rejected twice, with the same feedback: “One or more of your test cases did not produce correct results. Please re-run all submitted test cases and align tool behavior/output with the documented expected outcomes. Ensure the same test cases pass consistently on both ChatGPT web and mobile. Update either implementation or tests so results are reliable and unambiguous.”
We’ve tested locally in Web and mobile and all our test cases work as intended, for multiple accounts (we tried with accounts that had not used the app before in case memory affected). The only thing that does not call the app is when you have the app listed and connected within your account’s Apps but you do not manually choose/call it e.g. organic call by the LLM.
But, this behaviour does not raise any other published apps we tested e.g. if I prompt on ChatGPT: “Use the TripAdvisor app to find me a hotel in London tonight” which is an app I have listed and connected within my Apps, it still does not call it. It searches the web instead. It needs the @ or for the user to choose it by clicking + > More > App name. And this behaviour happens with X amount of apps we tried.
Right now, our test cases work properly when you choose the App as a connector or you mention it directly with @ in the chat text input but we still got the same rejection feedback twice.
Has this happened to anyone else? We tried appealing to hear from OpenAI but no response yet.