Plugins not following openapi.yaml specs for GET and POST endpoints?

[yawnxyz]

I’m running into something bizarre — the ChatGPT plugin doesn’t seem to be reading the YAML file correctly, and tries to hit my POST endpoint with a GET request (and trying to send json data along with it), even though the POST endpoint is clearly specified. This only happens with my Replit version; if I switch to my localhost version (same code), it’s not a problem.

I had to resort to adding a GET version w/ a query parameter to make it work.

Here’s the yaml:

openapi: 3.0.1
info:
  title: "BabyAGI localhost"
  version: 'v1'
  description: "Plugin for running autonomous agents to complete a final goal using BabyAGI. To use, first set the objective, then use add the appropriate tasks to achieve the goal. The goal is achieved by running the agent. Then call Execute next task until all tasks are done. Verify tasks are dne by getting the task list. Plugin has access to real-time data and internet search results."
servers:
  - url: http://localhost:8002
  - url: https://101-babyagi-plugin.janzheng.repl.co/
paths:
  /set_objective:
    post:
      summary: "Set the objective"
      operationId: "setObjective"
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: "object"
              properties:
                objective:
                  type: "string"
      responses:
        200:
          description: "Objective set successfully"
          content:
            application/json:
              schema:
                type: "object"
                properties:
                  status:
                    type: "string"
    get:
      summary: "Set the objective with a query parameter"
      operationId: "setObjective_get"
      parameters:
        - name: "objective"
          in: "query"
          required: true
          schema:
            type: "string"
      responses:
        200:
          description: "Objective set successfully"
          content:
            application/json:
              schema:
                type: "object"
                properties:
                  status:
                    type: "string"
  /add_task:
    post:
      summary: "Add a new task"
      operationId: "addTask"
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: "object"
              properties:
                task_name:
                  type: "string"
      responses:
        200:
          description: "Task added successfully"
          content:
            application/json:
              schema:
                type: "object"
                properties:
                  status:
                    type: "string"
    get:
      summary: "Add a new task with a query parameter"
      operationId: "addTask_get"
      parameters:
        - name: "task"
          in: "query"
          required: true
          schema:
            type: "string"
      responses:
        200:
          description: "Task added successfully"
          content:
            application/json:
              schema:
                type: "object"
                properties:
                  status:
                    type: "string"
  /get_task_list:
    get:
      summary: "Get the list of tasks"
      operationId: "getTaskList"
      responses:
        200:
          description: "Task list retrieved successfully"
          content:
            application/json:
              schema:
                type: "object"
                properties:
                  task_list:
                    type: "array"
                    items:
                      type: "string"
  /execute_next_task:
    post:
      summary: "Execute the next task and update the task list"
      operationId: "executeNextTask"
      responses:
        200:
          description: "Task executed successfully and task list updated"
          content:
            application/json:
              schema:
                type: "object"
                properties:
                  task_id:
                    type: "integer"
                  task_name:
                    type: "string"
                  result:
                    type: "string"
                  new_tasks:
                    type: "array"
                    items:
                      type: "object"
                      properties:
                        task_name:
                          type: "string"
    get:
      summary: "Execute the next task and update the task list"
      operationId: "executeNextTask"
      responses:
        200:
          description: "Task executed successfully and task list updated"
          content:
            application/json:
              schema:
                type: "object"
                properties:
                  task_id:
                    type: "integer"
                  task_name:
                    type: "string"
                  result:
                    type: "string"
                  new_tasks:
                    type: "array"
                    items:
                      type: "object"
                      properties:
                        task_name:
                          type: "string"

[dlaytonj2]

I haven’t tried a Post endpoint yet but I found this in the documentation:

“ When a user asks a relevant question, the model may choose to invoke an API call from your plugin if it seems relevant; for POST requests, we require that developers build a user confirmation flow.“

If you look at the To-do example that is what they do. Still, it doesn’t explain why it works locally but not remotely.

[yawnxyz]

Right, and preferably it should follow what’s GET and POST endpoints are available to it as specified in the yaml file, but alas, it makes up endpoints that don’t exist…

[ruv]

Looks like your summary and descriptions are fairly generic. Try giving it more descriptive text.

[yawnxyz]

Hold on, when given only a POST endpoint, it’s not smart enough to understand there isn’t a GET endpoint available, and you have to add more descriptive text to convince it that there’s only a POST endpoint? Then what’s the point of using OpenAPI spec?

[bruce.dambrosio]

Have you tried asking gpt to describe the endpoints? I often find its own description varies from what I thought I specified, even though fastapi or hypercorn serves the api as specified in the yaml.

[yawnxyz]

No, that’s a good point — I’ll try doing that!

It’s just weird how you can clearly specify “use POST endpoint” and it’ll still try to send JSON data to the same endpoint as a GET instead of POST request, which will fail, and it’ll get confused as to why.

My “dumb” way to get around this was creating a GET endpoint that takes URL variables just in case the plugin AI sometimes decides to go that route…

[wfhbrian]

Have you checked out the “Prompt for ChatGPT” in the devtools? It might be that the auto-generated prompt based on your OpenAPI spec is confusing ChatGPT.

I was curious so I tested out the OpenAPI spec you shared above. This is the “Prompt for ChatGPT” that was generated:

// Set the objective
type setObjective = (_: {
objective?: string,
}) => any;

// Set the objective with a query parameter
type setObjective_get = (_: {
objective: string,
}) => any;

// Add a new task
type addTask = (_: {
task_name?: string,
}) => any;

// Add a new task with a query parameter
type addTask_get = (_: {
task: string,
}) => any;

// Get the list of tasks
type getTaskList = () => any;

// Execute the next task and update the task list
type executeNextTask = () => any;

// Execute the next task and update the task list
type executeNextTask = () => any;

It looks ok, but it could probably be improved by adding descriptions to the component properties in your OpenAPI spec.

[yawnxyz]

Ok so the weirdest thing about all this is that if I use devtools, which requires localhost — everything works properly. All the correct endpoints are used.

If I upload my code to Replit and use Replit to host the plugin, it gets messed up and uses the wrong endpoints — and since it’s on Replit and not localhost, I can’t use devtools. I can’t replicate this weird behavior on localhost at all, as it’s always being read correctly! Hosting it on fly.io is also fine. But nothing about Replit should be wrong; I suspect there’s some kind of CORS thing happening in the background making it not properly read the YAML or something. But if I ask the plugin, in the chat box “what endpoints are available” it’ll tell it to me correctly…