Can a tool output type be any object type?
I am getting the below error when passing an array as a tool output-
BadRequestError: Error code: 400 - {‘error’: {‘message’: ‘1 validation error for Request\nbody → tool_outputs → 0 → output\n str type expected (type=type_error.str)’, ‘type’: ‘invalid_request_error’, ‘param’: None, ‘code’: None}}
Your tool output(s) needs to be a string.
It’s a response for GPT to use. If you want to return an object you can simply stringify it
What if my data is too large after stringifying it? Will it be truncated internally if due to some reason it exceeds some token limit?
Good question. Not sure if it would flatout reject it, or truncate. Time to find out 
If you don’t mind me asking, what are you trying to do & send back?
I need to send some data for analytics, e.g. calculate sum, avg, etc on an array
Getting this error when trying on an array of len 3 millions converted to string and sent as tool output –
InternalServerError: Error code: 500 - {‘error’: {‘message’: ‘The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID 890c71a662367dd3a9694897de7e71db in your email.)’, ‘type’: ‘server_error’, ‘param’: None, ‘code’: None}}
If you send to much input to the AI in any form, you will get a context length error.
This model’s maximum context length is 8191 tokens, however you requested 16296 tokens (16296 in your prompt; 0 for the completion). Please reduce your prompt; or completion length.
So my scenario is, I want to query a DB and then perform aggregations on it, e.g. return total number of orders will typically retrieve all orders and then perform a count on it. I have even defined a count method in the function tool section while assistant creation. But what the assistant is trying to do is calculate count, sum etc on its own assuming the entire input is available to it. It is not suggesting the count method. Not able to figure out a way for it to suggest the count method.
I cannot assist with assistants. They are an early beta product. They do their own thing internally outside of our control and without documentation.
No worries, I appreciate the support. Thanks
You’re sending data for GPT to perform calculations on?
I’m confused why you are sending an array with a length of 3 million. Are your values numbers?
Ideally I want assistant to identify the aggregation function (sum or count) , but instead it is expecting me to submit the values in tool output so that it can perform aggregation.
e.g. Given a query - return sum of all orders
Step 1 - assistant identifies to call get_orders function.
Step 2 - not sure how to proceed. At the moment I am submitting str(array of all orders)
Step 3 - gpt returns the sum or count whatever is expected.
Problem is I am not sure how to proceed with step 2 and 3 such that assistant suggests to call the sum or count function so that I do not have to send the large data to gpt for processing and my client code does the job.
I may be misunderstanding but is there a reason you are not just performing these calculations in the back-end and returning the results to GPT? LLMs cannot math well.
You are trying to determine if the array of strings is to be counted or summed? I’m sorry but I am completely confused.
Let’s try some psuedocode
fn get_orders(): string[] => db.retrieve("orders.*")
Your flow is “User -(unstructured query)-> GPT -(function call(get_orders)-> DB -(results)-> GPT -(perform_work(?) & respond) → User”
??
Why are you not doing
fn aggregate_orders(type: AggregateType): number => case switch blah blah blah
“User -(unstructured query)-> GPT -(function call(aggregate_orders(“sum”)-> DB -(results)-> GPT -(respond) → User”
share some of your function templates to be able to help better. The descriptions you put in the template are ‘mini prompts’ that will be used to decide on which functions to call. This goes for the description of the function itself but also for the fields that your define.
I wanted to the keep the functions granular. e.g. there will be aggregate on many entity types like orders, products, etc. Is there no way to get a sequence of function call suggestions from gpt like -
“User -(unstructured query)-> GPT -function call(get_orders)->perform work-> GPT -function call(sum)->(perform_work & respond) ->GPT-> User”?
I had a tangential question here. Are function templates injected as prompts (as context) along with the query? How are function templates stored/indexed? Is there a limit to the number of functions we can define in the assistant? I am new to LLMs, apologies if the questions are too basic. Thanks in advance.
Function definitions are done in the OpenAI backend - where you create your Assistant as well. (Or programmatically.)
https://platform.openai.com/docs/guides/function-calling
I just edited my question, I used the term function definition instead of function templates. Please review. Apologies again.
Yes - I understood. The definitions - ‘the templates’ CAN be entered straight in the backend. Not super easy (you have to paste JSON). So can be done programmatically too. Here’s and example from one of mine:
{
"name": "webScrape",
"description": "Get the text content of a webpage if 'ignore links' is true, links will be removed from the text",
"parameters": {
"type": "object",
"properties": {
"url": {
"type": "string",
"description": "The URL of the website to scrape"
},
"ignore links": {
"type": "boolean",
"description": "Ignore links in the text"
},
"max length": {
"type": "integer",
"description": "Maximum length of the text to return"
}
},
"required": [
"url",
"ignore links"
]
}
}
Yes, I am aware of the usage of function tool in our backend, but wanted to understand how does the assistant handle all the function templates. How does the assistant internally store the templates, does it internally maintain a vectorDB or indexing etc?