Plaintext schemas for chat functions

API
1

Much of the time the model will attempt to send very invalid JSON because it wants to write out very clean easy to read code. Here’s an example function that accepts graphql as the first argument, as a str.

import requests
import json

def pokemon_graphql(query: str):
    '''Query the PokeAPI with GraphQL. Provides up to date information.'''
    headers = {'Content-Type': 'application/json'}

    # Define the url of the GraphQL API
    url = "https://beta.pokeapi.co/graphql/v1beta"

    # Make the POST request to the GraphQL API
    response = requests.post(url, json={'query': query}, headers=headers)

    # Print the response as JSON
    return json.loads(response.text)

schema = {
  "name": "pokemon_graphql",
  "description": "Query the PokeAPI with GraphQL. Provides up to date information.",
  "parameters": {
    "type": "object",
    "properties": {
      "query": {
        "type": "string"
      }
    },
    "required": [
      "query"
    ]
  }
}

Sometimes it gets it very right.

image
image815×811 52.1 KB

Other times it gets it very wrong, though very readable!:

image
image814×761 49.2 KB

What I wish I could do is indulge the model by setting the parameters schema to a literal string.

def pokemon_graphql(query: str):
    '''Query the PokeAPI with GraphQL. Provides up to date information.'''
    headers = {'Content-Type': 'application/json'}

    # Define the url of the GraphQL API
    url = "https://beta.pokeapi.co/graphql/v1beta"

    # Make the POST request to the GraphQL API
    response = requests.post(url, json={'query': query}, headers=headers)

    # Print the response as JSON
    return json.loads(response.text)

schema = {
    'name': 'pokemon_graphql',
    'description': 'Query the PokeAPI with GraphQL. Provides up to date information.',
    
    # What if we could accept just a plain string?!
    'parameters': { "type": "string" }
}

c = Conversation(
    model="gpt-3.5-turbo-0613",
)
c.register(pokemon_graphql, json_schema=schema)
c.submit(
    system("You are allowed to debug and keep trying if your graphQL queries fail."),
    "Tell me up to date information about the pokemon named murkrow"
)

However, this errors with

InvalidRequestError: Invalid schema for function 'pokemon_graphql': schema must be a JSON Schema of 'type: "object"', got 'type: "string"'.

Something similar happens for allowing it to submit code or really anything where the one and only argument is a string. Perhaps we could indulge the model and expand the JSON schema that is allowed in the parameters schema. Maybe the field isn’t parameters for that case? Not sure! Not really my call either but I’d love to be able to really crank up what we can do with strings into functions if it can do plaintext as input.

2 Likes
2

I ran into a similar issue with a hallucinated python function. I of course had to indulge it in chatlab.

2 Likes
3

I think the way openai is doing it with the open api schema isn’t that useful. The schema is an internal representation and they should keep it that ways. I have been working on an API automation solution using GPT and am approaching it little differently. Would be happy to share my approach to the openAi folks if they want.

1 Like
4

A few tips on functions:

I’ve seen a lot of weird going on with the gpt-3.5-turbo-16k-0613 model. Try switching to gpt-3.5-turbo-16k or gpt-3.5-turbo, and use the following parameters.

{
  "messages": ...,
  "model" : "gpt-3.5-turbo-16k",
  "temperature" : 0.0,
  "n" : 1,
  "top_p" : 0.0,
  "frequency_penalty" : 0.0,
  "presence_penalty" : 0.0
}

The issue the OP is having is on the JSON schema itself.

[
  {
    "name": "summarize_dialogue_by_person",
    "description": "Summarizes the text in the quotes to be less than 25 words",
    "type": "object",
    "parameters": {
      "type": "object",
      "properties": {
        "quotes": {
          "description": "The quotes to summarize",
          "type": "array",
          "items": {
            "description": "Shorten the text in the quote to be less than 25 words",
            "type": "object",
            "properties": {
              "quote": {
                "description": "Shorten the text of the quote to be less than 25 words",
                "type": "string"
              },
              "person_name": {
                "description": "The name of the person speaking",
                "type": "string"
              }
            },
            "required": [
              "quote",
              "person_name"
            ]
          }
        }
      },
        "required": [
            "quotes"
        ]
    }
  }
]

parameters must be of type object. I hope that helps!

5

Yes. The parameters are of type object. What I’m wondering about is… what if a single parameter could be a string and passed as is?