Does anyone have experienced tool calling inconsistency in full screen mode?
Our usecase is users can find flight prices for different destinations and by clicking on it we display a map in fullscreen. Once in fullscreen, we want to let users search for more destinations or change criteria in the nav bar “ask jinko”. However, we experienced the tool calling at this step is not working properly or it’s very inconsistent. When we re-select in the nav bar our app via the launcher it works properly.
Hi @Kevin_Gauthier , when you say “Reselect the app”, what do you mean? Do you mean hitting the “+” and finding the app and selecting it? Or do you mean interacting with your widget and putting focus on it? What sort of tool call should it be making? What’s the source of the tool call, is it manual (via code) or is it something the model is supposed to call in this case?
I mean hitting the “+” and finding the app and selecting it. The tool to be called is exactly the same as the one triggered to open the widget in fullscreen.
The bad behaviour is if the user do not click on the launcher and select Jinko, it will not trigger a new tool call and will not refresh the widget. Whereas if the user clicks on the launcher and select Jinko, it’s working properly.
From the user perspective i will not expect to have to re-select Jinko once i’m in a fullscreen mode on the app. It should be automatically triggered by the model.
I’ve had this exact problem as well. It’s especially bad when it’s an enterprise app and when the user ask follow up questions in fullscreen mode, they get responses from competitors because it’s no longer using the app as context.
Absolutely agree with you here, I think the default behavior should be, when a widget is in full screen, and the user uses the composer to prompt any further, the tool call should almost be “guaranteed” as this is 90% of the time why the user is interacting with the prompt in this scenario.
