Improve multiple function calling using API

API
, ,
1

Hey folks!

I’m building a function calling prompt, but the results are terrible.

Prompt

Please select the best function to answer user questions and context. Follow this instructions:

  1. Only use functions that you have access to;

  2. Consider the user questions and history to understand which tool should be used;

  3. IMPORTANT: You can call more than one function, understand all the context, and consider the best tools to answer user questions completely;

  4. History is only necessary for context; the question is most important.

You have access to these tools:

Functions structure

GetDeliveryStatusForWorkItems

Get the delivery status (whether late or on track) for work items. Response structure: {{“workItemsWithDeliveryStatus”: [{“id”: “string”, “key”: “string”, “title”: “string”, “actualStatus”: “string”, “assignedTo”: “string”, “leadTimeToEnd”: “number”, “leadTimeUsed”: “number”, “percentageLeadTimeAlreadyUsed”: “number”, “leadTimeToEndWithLeadTimeAlreadyUsed”: “number”, “percentageLeadTimeExceeded”: “number”, “isLate”: “boolean”, “onTrackFlag”: “string”}]}}

GetWorkItensTool

Get work itens by keys or ids passed as parameters. Response structure: { “WorkItems”: [{“key”:“string”,“id”:“string”,“name”:“string”,“description”:“string”,“created”:“string”,“updated”:“string”,“changelog”:[{“id”:“string”,“createdAt”:“string”,“movements”:[{“field”:“string”,“fromColumnId”:“string”,“fromColumnName”:“string”,“toColumnId”:“string”,“toColumnName”:“string”}]}],“workItemCreatedAt”:“string”,“columnName”:“string”,“priority”:“string”,“flagged”:“boolean”,“assignee”:{“accountId”:“string”,“userName”:“string”},“workItemType”:{“name”:“string”,“id”:“string”,“description”:“string”,“subtask”:“boolean”},“status”:{“name”:“string”,“id”:“string”,“statusCategory”:{“name”:“string”,“id”:“number”}}}]}

Functions JSON Schema

* 0: { "type": "function", "function": { "name": "GetColumnsConfigTool", "description": "Get the Id and Name for columns(To Do, WIP, and Done) from a board along with their respective order. Response Structure: \"{\"wipColumnsAndDoneColumns\":[\"string\"],\"columnsConfig\":{\"allColumns\":[{\"id\":\"string\",\"name\":\"string\",\"order\":\"number|null\",\"column\":\"string\"}],\"wipColumns\":[\"string\"],\"doneColumns\":[\"string\"],\"todoColumns\":[\"string\"]}}\"", "parameters": { "type": "object", "properties": {}, "additionalProperties": false, "$schema": "http://json-schema.org/draft-07/schema#" } } }
* 1: { "type": "function", "function": { "name": "GetWorkItemsDeliveryStatusTool", "description": "Get the delivery status (whether late or on track) for work items. Response structure: {{\"workItemsWithDeliveryStatus\": [{\"id\": \"string\", \"key\": \"string\", \"title\": \"string\", \"actualStatus\": \"string\", \"assignedTo\": \"string\", \"leadTimeToEnd\": \"number\", \"leadTimeUsed\": \"number\", \"percentageLeadTimeAlreadyUsed\": \"number\", \"leadTimeToEndWithLeadTimeAlreadyUsed\": \"number\", \"percentageLeadTimeExceeded\": \"number\", \"isLate\": \"boolean\", \"onTrackFlag\": \"string\"}]}}\n This function need a get workItem function before.", "parameters": { "type": "object", "properties": { "parameters": { "type": "array", "items": { "type": "object", "properties": { "workItems": { "type": "array", "items": { "type": "string", "description": "Work Item Id" }, "description": "value extracted from a text and returned as a parameter, examples of what the value looks like (GE-18, KDZ-20, NT-10, HYPER-44, APP-538)" } }, "additionalProperties": false } } }, "required": [ "parameters" ], "additionalProperties": false, "$schema": "http://json-schema.org/draft-07/schema#" } } }
* 2: { "type": "function", "function": { "name": "GetWorkItensTool", "description": "Get work itens by keys or ids passed as parameters. Response structure: { \"WorkItems\": [{\"key\":\"string\",\"id\":\"string\",\"name\":\"string\",\"description\":\"string\",\"created\":\"string\",\"updated\":\"string\",\"changelog\":[{\"id\":\"string\",\"createdAt\":\"string\",\"movements\":[{\"field\":\"string\",\"fromColumnId\":\"string\",\"fromColumnName\":\"string\",\"toColumnId\":\"string\",\"toColumnName\":\"string\"}]}],\"workItemCreatedAt\":\"string\",\"columnName\":\"string\",\"priority\":\"string\",\"flagged\":\"boolean\",\"assignee\":{\"accountId\":\"string\",\"userName\":\"string\"},\"workItemType\":{\"name\":\"string\",\"id\":\"string\",\"description\":\"string\",\"subtask\":\"boolean\"},\"status\":{\"name\":\"string\",\"id\":\"string\",\"statusCategory\":{\"name\":\"string\",\"id\":\"number\"}}}]}", "parameters": { "type": "object", "properties": { "parameters": { "type": "object", "properties": { "workItemsIds": { "type": "array", "items": { "type": "string", "description": "WorkItemId. identifier for a work item. Examples: GE-18, KDZ-20, NT-10, HYPER-44, APP-538. Each ID represents a specific work item in a project or system." } } }, "additionalProperties": false, "description": "Array of paramaters mentioned in text. You can use chat history to provide more context and get parameters" }, "operation": { "type": "array", "items": { "type": "string", "enum": [ "ByWeek", "inWIP", "ByIds", "Last24hours", "ByTypes" ], "description": "The inner filter for Work Items." } } }, "required": [ "parameters", "operation" ], "additionalProperties": false, "$schema": "http://json-schema.org/draft-07/schema#" } } }

User question

What are the delivery dates for tasks that are in progress?

Expected result

functions GetWorkItensTool and GetDeliveryStatusForWorkItems.

Actual result

function GetWorkItensTool

Would you happen to have any tips about how to improve it? Is there any better model than openAI GPT-4-turbo for it?

1 Like
3

Hi there,. I have also been working on the same problem for a few days. Using Pydantic helped a lot. You could try that.

I use something like this
def classify(self) → List[Dict[str, str]]:

to first get a list of intents, and then call separate functions for each of the user intent

Pydantic might also help you get the right format for the response