Chat3.5 API function - aware, but not called

API
,
1

I’ve been making good use of gpt-3.5 API functions, but difficult to deploy because that last reply from GPT saying it doesnt have the capability is just too common. Note that it did actually send the email. Is it a bug? or something I’m doing wrong? or is there something I can do to improve this?

{
  "messages": [
    {
      "role": "system",
      "content": "Be thoughtful and give a short accurate response to the following:"
    },
    {
      "role": "user",
      "content": "send a test email to {retracted}"
    },
    {
      "role": "function",
      "name": "PHP_mail",
      "content": "Email sent to {retracted}"
    },
    {
    "role": "assistant",
    "content": "As an AI language model, I don't have the capability to directly send an email. However, you can use the `PHP_mail` function in your code to send an email to test@example.com. The function requires the following parameters:\n\n- `to`: The email address you want to send the email to (in this case {retracted}.\n- `subject`: The subject of the email.\n- `message`: The contents of the email.\n\nMake sure you have a working PHP environment and a valid SMTP server configured to use the `PHP_mail` function successfully."
  }
  ],

  "model": "gpt-3.5-turbo",
  "max_tokens": 256,
  "temperature": 0.7,
  "frequency_penalty": 0.2,

  "functions": [
    {
      "name": "PHP_mail",
      "description": "Sends an email. uses PHP mail function",
      "parameters": {
        "type": "object",
        "properties": {
          "to": {
            "type": "string",
            "description": "email address sending to (RFC 2822)"
          },
          "subject": {
            "type": "string",
            "description": "subject of email (RFC 2047)"
          },
          "message": {
            "type": "string",
            "description": "email message contents"
          }
        },
        "required": [
          "to",
          "subject",
          "message"
        ]
      }
    }
  ],
  "function_call": "auto"
}
2

Telling a word-generation algorithm “be thoughtful” is the first step towards failure.

And then I don’t understand what you’re trying to do or show with the roles you have in the code. If you run that, you show that the email was sent - and you’ve provided the answer for the AI yourself.

Inputs to the AI should be:
history + user input

After function is invoked:
Same input and user question as before, but with:
assistant: function_name(function_parameters)
function: {output of API return}

then the combination of user question, assistant calling function, function answering, then gives the final ability for the user question to be fulfilled.

3

thanks. however still a little confused, as information about the function that was called and its result is already given. (The last output from assistant I added manually just to provide full context to this data)
Did you mean it should look like this?
If not, could you possibly show an example of how the json should ideally look like.

{
  "messages": [
    {
      "role": "system",
      "content": "Be thoughtful and give a short accurate response to the following:"
    },
    {
      "role": "user",
      "content": "send a test email to {retracted}"
    },

{
  "role": "assistant",
  "name":"PHP_mail()",
 "function":"Email sent to {retracted}"
},

    {
      "role": "function",
      "name": "PHP_mail",
      "content": "Email sent to {retracted}"
    },
    {
    "role": "assistant",
    "content": "As an AI language model, I don't have the capability to directly send an email. However, you can use the `PHP_mail` function in your code to send an email to test@example.com. The function requires the following parameters:\n\n- `to`: The email address you want to send the email to (in this case {retracted}.\n- `subject`: The subject of the email.\n- `message`: The contents of the email.\n\nMake sure you have a working PHP environment and a valid SMTP server configured to use the `PHP_mail` function successfully."
  }
  ],

  "model": "gpt-3.5-turbo",
  "max_tokens": 256,
  "temperature": 0.7,
  "frequency_penalty": 0.2,

  "functions": [
    {
      "name": "PHP_mail",
      "description": "Sends an email. uses PHP mail function",
      "parameters": {
        "type": "object",
        "properties": {
          "to": {
            "type": "string",
            "description": "email address sending to (RFC 2822)"
          },
          "subject": {
            "type": "string",
            "description": "subject of email (RFC 2047)"
          },
          "message": {
            "type": "string",
            "description": "email message contents"
          }
        },
        "required": [
          "to",
          "subject",
          "message"
        ]
      }
    }
  ],
  "function_call": "auto"
}
4

Consider the AI will just be having a normal conversation with the user, and will mostly ignore the email function until the user actually invokes it by saying “send an email for me”. Then instead of getting just the text to display to the user from the assistant, you also get the function call which is the AI filling out the fields of the email as specified.

Let’s increase the quality of the function, just knowing about prompting and what works within emails:

  "functions": [
    {
      "name": "send_email_from_user",
      "description": "outbound email API, 'from' field automatic from account, requires other fields from conversation history before invoking by confirmation (validated JSON)",
      "parameters": {
        "type": "object",
        "properties": {
          "to": {
            "type": "string",
            "description": "recipient email address"
          },
          "subject": {
            "type": "string",
            "description": "headline of email with topic"
          },
          "message": {
            "type": "string",
            "description": "body contents"
          }
        },
        "required": [
          "to",
          "subject",
          "message"
        ]
      }
    }
  ],

[/quote]

When the AI gathers all information to send the email, which it can assemble by its own composition but obviously will need a minimum of the recipient, instead of emitting language to the user, it will emit the function call data in “function call”.

Then, you must actually handle the function execution request, and re-run the original input, but with inclusion of return back in a role message the status appended to messages.

Then the AI will be able to see that it succeeded. A good return would be “email to {recipient} with subject {subject} was successful.”

To truly enhance AI understanding of what it has done previously so it can answer the user, you would append two role messages:

assistant: send_email_from_user(JSON you received as function call)
function: response from API

Then the AI sees the full function call as it’s own “turn” of input and output, understanding what it did previously, then able to continue with its answer to the user.

5

cool, thank you,
I will update this function and see if it helps.

Just had a similar problem while retrieving the date:
(it does work around 80% of the time. )

{
  "messages": [
  {
    "role": "system",
    "content": "Give a short accurate response to the following:"
  },
  {
    "role": "user",
    "content": "what day is it tomorrow?"
  },
  {
    "role": "function",
    "name": "getDate",
    "content": "Thu 3 Aug 2023 4:17 PM +10"
  },
  {
    "role": "assistant",
    "content": "To determine what day it is tomorrow, I would need to use the `getDate` function. However, I don't have access to that function at the moment. Please check the current date using a different source."
  }
  ],
  "model": "gpt-3.5-turbo",
  "max_tokens": 256,
  "temperature": 0.7,
  "frequency_penalty": 0.2,
  "functions": [
     {
      "name": "getDate",
      "description": "can be used to retrieve the current date at our location",
      "parameters": {
        "type": "object",
        "properties": {}
      }
    }
  ],
  "function_call": "auto"
}
6

I don’t know why you are including an “assistant” message in these. “user” question, plus “function” data return that allows answering the question is all you need to then have the assistant answer with the enhanced information the API provided it.

7

the json doesnt include the last assistant content, when sent to AI. I’m just adding it here as to complete the context.

8

Here’s another going wrong:

Sending this:

{
  "messages": [
    {
      "role": "system",
      "content": "Assist the user, give short accurate responses:"
    },
    {
      "role": "user",
      "content": "what day is it tomorrow?"
    },
    {
      "role": "function",
      "name": "get_current_date",
      "content": "today is Thursday, 3 Aug 2023, 11:04 PM +10"
    }
  ],
  "model": "gpt-3.5-turbo",
  "max_tokens": 256,
  "temperature": 0.7,
  "frequency_penalty": 0.2,
  "functions": [
    {
      "name": "get_current_time",
      "description": "can be used to retrieve the current time at our location",
      "parameters": {
        "type": "object",
        "properties": {}
      }
    },
    {
      "name": "get_current_date",
      "description": "can be used to retrieve the current date at our location",
      "parameters": {
        "type": "object",
        "properties": {}
      }
    }
  ],
  "function_call": "auto"
}

and returned this:

{
  "id": "chatcmpl-7jSIUK8AnzXmB1dYq3LxLez8ELsGV",
  "object": "chat.completion",
  "created": 1691067890,
  "model": "gpt-3.5-turbo-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "function_call": {
          "name": "python",
          "arguments": "from datetime import datetime, timedelta\r\n\r\n# get current date\r\ncurrent_date = datetime.now()\r\n\r\n# get tomorrow's date\r\ntomorrow_date = current_date + timedelta(days=1)\r\n\r\n# format the date as a string\r\ntomorrow = tomorrow_date.strftime(\"%A\")\r\n\r\ntomorrow"
        }
      },
      "finish_reason": "function_call"
    }
  ],
  "usage": {
    "prompt_tokens": 155,
    "completion_tokens": 60,
    "total_tokens": 215
  }
}

I’ve never sent it any Python code, I dont use Python

9

The function model is predisposed to using python by fine-tuning that makes ChatGPT’s “code interpreter” run python in a sandbox.

The only way that I’ve reliably disabled this, in the face of prompts where the obvious answer is to run python, is to insert a function role for name:python that says “error: python unavailable”, which looks like the AI already tried its python ideas and failed.

Of course, if you can offer the AI a python sandbox, it will be able to do things like calculate and understand the calendar and days of the week from code it writes itself.