How to elicit / capture user feedback?

GPT builders Plugins / Actions builders
1

When building a plugin, is there any way to get user feedback on the quality of responses? e.g. similar to (or actually getting the results of) openai’s thumbs up/down?

Or any access to stats about re-rolls of responses?

I’m not sure how to go about measuring the effectiveness or quality of my plugin’s responses.

5 Likes
2

We don’t have anything around this right now, but we should!

3 Likes
3

A standardised endpoint that we (plugin devs) can provide in the openapi spec would be ideal I think. Then openai can hit that whenever a user provides a thumbs up/down. Something like this:

class UserFeedback(BaseModel):
    """This model represents the user feedback on a specific interaction with the plugin."""
    request: Any = Field(..., description="The request from openai")
    response: Any = Field(..., description="The response provided to openai")
    openai_response: str = Field(..., description="The response from openai to the user")
    feedback: Literal["thumbs_up", "thumbs_down"]

@app.post("/user-feedback")
def user_feedback(user_feedback: UserFeedback):
    """
    This endpoint is called when the user provides feedback on a specific interaction with the plugin.
    """
    return controller.user_feedback(user_feedback)
1 Like
4

Something that’s maybe worth emphasising in the above: Considering the non-deterministic nature of the whole system, it would be really great to have the openai response that the user sees included along with any feedback. It seems there are a few different levers that influence this, so being able to experiment on how best to influence this is going to be important.

2 Likes