How do you deal with long queries?

GPT builders Plugins / Actions builders
,
1

Hi everybody,

I am building a plugin that relies on an API that produces the final result typically in 30s - 2 mins. I have two options for implementing a wrapper API:

  • Immediately return a task_id while the job gets executed, wait some time, and perform a call to get_output(task_id).
  • Wait until we have the final result and then return it.

What will be the best approach for dealing with such a long response? Will the plugin prompt timeout if I exceed a specific amount of time? How can I teach chatGPT to perform multiple calls for starting the job, polling it, and getting the output, all in one single step?

1 Like