When I use a custom action to fetch data from an API endpoint, the data is returned instantly as JSON, so why isn’t the data displayed instantly? Instead, Chat GPT types out each line one by one, which takes forever. Is there any prompt you can give it to output data from an endpoint faster?
Because the action response doesn’t output to the chat stream. It outputs to the context which the model then reads and reports to the user via the chat stream.
There is nothing you can do to speed this up.
1 Like
I wish there were a way to bypass this default functionality is some contexts.