How to know whether GPT I built on GPT store is working as expected?

[hshahid]

Hi,
I just created a GPT and I am not sure how to test the responses whether they are consistent for every user. In addition, is there a way to see where the conversation is failing or what’s the quality of overall conversation (like how many thumbs up or down)
thanks.

[jlfuertes]

Hi,
As far as I know, the only visible metric is the number of chats opened. I fully agree that this is a significant limitation when it comes to improving our models. I hope OpenAI provides more visibility regarding the performance of GPTs if they want to quickly develop useful tools based on GPT-4.

[matcha72]

One option is adding analytics to your GPT via 3rd party platform which can log user messages and your GPT response which you can analyze further

[peter.malinverni]

Seems relevant to what you’re asking for

Logging user conversations

Howdy! I’m building a tool that allows GPT builders (and anyone making a GPT wrapper using the API) to see actual user conversations, and get statistics like average messages per conversation, number of conversations ended after a single message, rate of positive (versus negative) conversations, and much more. Any interest in such a tool? Any feedback on the idea? I’ve set up a waitlist so you can be notified when the product is released!