I am building a plugin that relies on an API that produces the final result typically in 30s - 2 mins. I have two options for implementing a wrapper API:
- Immediately return a task_id while the job gets executed, wait some time, and perform a call to get_output(task_id).
- Wait until we have the final result and then return it.
What will be the best approach for dealing with such a long response? Will the plugin prompt timeout if I exceed a specific amount of time? How can I teach chatGPT to perform multiple calls for starting the job, polling it, and getting the output, all in one single step?