Wanting to see the queries and responses from my CustomGPT users

GPT builders
1

I am creating a CustomGPT that is focused on US DOT and FMCSA compliance for the US trucking industry.
I have uploaded all my specific data via PDF and started building out instructions. In testing a basic question that was kinda a “Gotcha” question. Many people misunderstand several parts of the answer. The response was correct except for 1 part.
When I am ready to test this with a handful of clients, how do I see the questions and responses so that I can fine tune the GPT?

1 Like
2

This promise is seen in your GPTs.

image

(Even when the “builder” is OpenAI - and thus they can)

1 Like
3

Hi @bigbadbill and welcome to the community!

If I understood correctly, you are creating a GPT for compliance based queries, and you would like to see the queries and the responses that your GPT generates, so you can improve it?

Unfortunately, this is not possible with CustomGPT. Unless of course you have an enterprise contract with OpenAI and you can get this kind of information via Compliance API.

But assuming you don’t have any fancy contracts, here are your options:

  • In your CutomGPT builder, supply a mailbox/email so users can email you feedback with queries and responses
  • Take a small group of beta users, get them to use it, and conduct interviews/surveys with them
  • Build a custom site that uses Assistants API instead - and log the queries and responses behind the scenes in a db - this requires some sophistication with software development of course - if you decide to go that route I’m sure there will be folks here that can help guide you.
1 Like
4

Using a technology that hallucinates is not something that should ever be used in something like compliance. Consider the liability when your gpt decides to make up its own rules and someone takes them as true. Please learn more about how this technology works before doing anything else.