How to Incorporate Third-Party Human Input into ChatGPT API Conversations After a Function Call?

API
,
1

I’m developing an application using the ChatGPT API where the assistant interacts with users to perform certain tasks via function calls. Occasionally, after a function call is made, a third-party human (e.g., a moderator or service representative) needs to review the request and provide feedback or additional information. I’m unsure how to properly integrate this third-party input back into the conversation between the user and the assistant.

Scenario:

  1. User: Requests a specific action or service.

  2. Assistant: Acknowledges the request and makes a function call to initiate the process.

  3. Function Call: Executes and returns a preliminary result (e.g., “Your request has been submitted”).

  4. Third-Party Human: Reviews the request and provides feedback (e.g., approval, denial, or alternatives).

  5. Assistant: Needs to relay this third-party feedback back to the user appropriately.

Constraints:

• Tool Calls Limitation: I can’t use a tool call to deliver the third-party input because tool calls and responses are sequential. Waiting for human interaction could disrupt the conversation flow, especially if the user sends additional messages during this time.

• Role Restrictions: The ChatGPT API supports only three roles: “system,” “user,” and “assistant.” Introducing a new role like “human_reviewer” results in an API error.

• Role Misinterpretation: Using the “user” role for the third-party input might confuse the model, as it assumes messages from the “user” role come from the original user.

• “Assistant” Role Misuse: Using the “assistant” role for the third-party input is inappropriate, as it should represent the AI assistant’s responses.

Options Considered:

Option A:

Attempting to introduce a new role (e.g., “human_reviewer”):

[
    {"role": "user", "content": "I'd like to request service X."},
    {"role": "assistant", "content": "Sure, I'll initiate that request for you."},
    {"role": "assistant", "tool_calls": [...]},
    {"role": "tool", "content": "..."},
    {"role": "assistant", "content": "Your request has been submitted."},
    {"role": "human_reviewer", "content": "Service X is unavailable. Optionally, consider service Y."},
    {"role": "assistant", "content": "Apologies, but service X is unavailable. Would you like to try service Y instead?"}
]

Issue: Introducing “human_reviewer” as a role is unsupported and causes an API error.

Option B:

Using the “system” role to provide third-party input:

[
    {"role": "user", "content": "I'd like to request service X."},
    {"role": "assistant", "content": "Sure, I'll initiate that request for you."},
    {"role": "assistant", "tool_calls": [...]},
    {"role": "tool", "content": "..."},
    {"role": "assistant", "content": "Your request has been submitted."},
    {"role": "system", "content": "Service X is unavailable. Optionally, consider service Y."},
    {"role": "assistant", "content": "Apologies, but service X is unavailable. Would you like to try service Y instead?"}
]

Concern: Uncertain if it’s appropriate to use the “system” role for this purpose.

Option C:

Using the “user” role with a “name” attribute to distinguish the third-party input:

[
    {"role": "user", "content": "I'd like to request service X."},
    {"role": "assistant", "content": "Sure, I'll initiate that request for you."},
    {"role": "assistant", "tool_calls": [...]},
    {"role": "tool", "content": "..."},
    {"role": "assistant", "content": "Your request has been submitted."},
    {"role": "user", "name": "human_reviewer", "content": "Service X is unavailable. Optionally, consider service Y."},
    {"role": "assistant", "content": "Apologies, but service X is unavailable. Would you like to try service Y instead?"}
]

Concern: The assistant might still interpret this as input from the original user.

Questions:

  1. What is the correct way to inform the assistant of third-party human input within the constraints of the ChatGPT API?

  2. Is using the “system” role appropriate for delivering this message to the assistant?

  3. Can I include a “name” attribute for the “user” or “system” role to differentiate the message source?

  4. Are there best practices for handling scenarios where external information needs to be fed back into the conversation after a function call?

Any guidance or examples on properly structuring the conversation to incorporate third-party human feedback would be greatly appreciated.

2

I would use
[SYSTEM]: The human reviewer has received the request… (I might call them “Supervisor”)

you can start a new run of the thread (much) later when the human has responded and send a new SYSTEM message then:

[SYSTEM]: The human reviewer says Yes.

Did that help?

3

It’s the Open AI API, not ChatGPT … that’s a different product (using the same models)

2 Likes
5

since this is not streaming by the nature of your use case, you can easily implement this using normal chat completions. let say, the user and the reviewer is connected to the running app (perhaps using web socket, like typical chat app, although the reviewer has different role to the user).

  1. when you send an inquiry, perhaps you want to disable the UI until you get a response.
  2. your inquiry is sent to the backend. if it does not invoke function calling, you just return back the response. your UI will then pickup the response, enable the UI and display it.
  3. if it invokes function calling, the tool response is sent to the reviewer. a popup message comes to alert in the reviewer’s UI. they perform their task and submit it back to the backend.
  4. your backend then sends back the response to your frontend, enables the UI and displays it.
3 Likes
7

You probably want to read this?

Your mileage with the assistant may vary but give it a go or switch to chat completions.

3 Likes