I am currently developing a ChatGPT plugin that integrates with an external service. However, the service endpoint I am using initially returns a queueID instead of the final response. The final response can be obtained later by using this queueID.
I am curious if this scenario can be supported within the ChatGPT plugin structure. If so, how can I manage this delayed response and queueID mechanism in my plugin implementation without redirecting the user to an external page? Is there a way to continuously check for the results until they become available or manage this asynchronous behavior within the plugin?
Any guidance or suggestions would be highly appreciated. Thank you in advance!
As far as I know, there is no way for a plugin to ‘initiate’ an interaction with the current chat session. Wish there was. I have been able to direct chatGPT to call a plugin after each user interaction, by injecting prompts in plugin responses, but I’m not sure whether that is intended behavior or will be supported long term.
Thank you for taking the time to respond to my question.
However, I don’t think I communicated my issue effectively. I understand that a plugin cannot initiate an interaction with the current chat session, but my concern is with how to handle delayed responses and queueIDs when interacting with an external service within the plugin structure.
Do you have any suggestions for how to handle this asynchronous behavior within a plugin?
I appreciate any help or guidance you can offer. Thank you again!
As I understand your problem, it seems to me the plugin, which runs in a server to provide an endpoint for chatGPT to invoke, is potentially always active - it could spawn a separate thread, for example, to check for completion. So now the problem is how to inform chatGPT. Two options: chatGPT could poll, or the endpoint could ‘push’ a notification. Option two is out for the reason I mentioned in my earlier response - there is no way for a plugin to ‘push’, chatGPT MUST initiate an interaction. Therefore you need to find a way to get chatGPT to poll. One way to do that is to ‘instruct’ chatGPT to call the plugin on a regular basis, for example, as suggested above. chatGPT is quite good at interpreting responses. I wouldn’t be surprised if, in an initial request, the agent returned something like ‘I have assigned this query ticket # n. I don’t have an answer yet, but I’ll let you know.’ then when chatGPT does another call, the plugin can response to that call, then append 'Oh by the way, I have an answer for ticket # n: …)
You would need to flesh out the response schema to handle both types of information, as well as the instructions on how and when to poll the plugin.
Again, my response above suggests any user interaction would be an appropriate time to poll, I don’t think chatGPT has any kind of internal clock or timers that can be set to trigger activation, although I haven’t explored that. Time to stop. A fascinating problem, but exploring further is something I can’t afford right now. Hope that helps some.
Disclaimer - I may sound like I know what I’m talking about, but I really don’t, the above may be a false start.