Assistant API get raw function output

My backend flow

  1. user creates thread
  2. user submits messages
  3. AI Assistant runs function if it chooses to
  4. Thread returned to FE

but for some of functions that are called by the assistant id like to render a UI component for them from the raw output of the function that has been called. The issue is then the function call is added to the thread as the natural language text and I want the raw json from the function is there a good way of doing this?

it be nice if they was added role:function when they have called a function in the thread

so main goal is to render UI components from the raw JSON output of AI function calls instead of displaying them as natural language text in the thread.

ideal thread would be

[
userMsg
assistantMsg
fncRes
userMsg
]

Assistants does not allow you to continue a thread without passing a function return and having the AI create a response to the user.

so can I make the function return just the raw output in the thread?

A real function is also not allowed to be seen “raw”. You would have to write your own special output format the AI can understand - and then not use any internal or external tools or you’d get a very confused AI.

Something like "

…you have a special function you can invoke for showing the user a form in their user interface instead of responding to the user with natural language. By producing a valid JSON response within four square brackets which contains a JSON object, the output will be sent to an API for interpretation of the function instead of being sent to the user.

// function json schema

Then your code that receives the assistant message can parse for forms, and not need to return a value unless you want. Also showing the ease of doing what you want on the completions endpoint, where you can utilize a function call as you wish and not need to return it.