Assitants API - unstable functions. Assistant making up dict keys

API
1

Hello, while trying to create multiuse CRUD functions on an assistant, I continuously get different function call arguments.

for an example:

both of the following two outputs have been created through the same prompt:
“please delete all emails that have been reached out to”

  1. Example:
    {"file_name":"contacts.csv","action_sequence":[{"action":"delete","condition":{"Was Reached Out To":true}}]})

  2. Example:
    {"file_name":"contacts.csv","action_sequence":[{"action":"delete","search_column":"Was Reached Out To","search_value":true}]}

The major issue is the assistant makes up dictionary key names, this of course becomes even worse when you make several conditions.

Has anyone had any experience with this?

2

Not with assistant functions in particular (reason why not relevant), but it’s a common manifestation of LLM use in general.

Here’s some facts:

  1. Given enough time, any LLM invlved task will eventually fail. No way ariound this.
  2. You can reduce the failure rate by:
    2.1. Using a stronger model
    2.2. Writing a better prompt
    2.3. Simplifying the problem (functions may complicate things)
    2.4. Reducing temperature and top_p.
    2.5. Bonus: try to create a convergent prompt - such that even if the model screws something up, it has an opportunity to recover (tough with assistants)

:slight_smile: