I’ve written a Custom GPT that makes calls to my API to answer users’ questions. The size of the data available to query is very small, but it updates several times an hour.
The GPT’s instructions are to parse the user query into the correct params to call my API with. Currently, I have given it very basic functionality. However, the GPT seems to be able to “do more” than I expected, answering queries that I didn’t specifically write instructions for (even though the API does have the functionality).
That got me thinking that maybe the GPT is, um “thinking”, and actually “deciding” to expand on its parsed version and grab all the data from my API at once. I am adding in better logging features on my API to see if that’s the case. It would be pretty cool if that’s happening.
Anyway, it also got me thinking that instead of my API providing a subset of the data each query, it simply returns all the data (again, it’s fairly small) and I give the GPT instructions to grab a new copy of the data every 15 minutes at minimum. My question is, will the GPT retain that data in context long enough to answer questions until the next time it grabs a snapshot of the data? Does that all sound reasonable?
GPTs cannot call API routes that you don’t specify in your schema. If you think it’s doing this, it’s probably hallucinating. (GPTs can browse pages on the public internet, so perhaps if your API routes are GET-only and don’t need auth it might be able to read them)
Yes, GPT-4-Turbo-Gizmo (the version that GPTs use) has a ~32k token context window. If your dataset is less than, say, 32kB, you should be able to fit it into the context window and answer a few questions about it. If it falls out of the context window, the GPT will just fetch it again.
Your GPT has no idea what time it is, so asking it to fetch ‘every 15 minutes at minimum’ doesn’t make sense from the model’s perspective. If it’s not critically important that the data is up to date, I wouldn’t bother prompting it, otherwise you could add some custom instructions telling the GPT that the data will frequently be stale and it should fetch new copies ‘regularly’ or ‘frequently’. It’ll get the idea, but to reiterate, there’s no way to specify that frequency.
I think a GPT can call API routes that aren’t specified in the schema, mine even tried to call an endpoint that doesn’t exist, earlier. I have one called Add_User_Message and it tried to call Add_GPT_Message, which does not and never has existed. I think it got confused because there is a Get_User_Messages and a Get_GPT_Messages, but to Add them it passes in a ‘speaker’ variable to Add_User_Message. After I pointed this out to it, it suggested I make my endpoints more intuitive, which was a valid point, but it can definitely attempt to call things not in its schema.
Both endpoints that I specify in the schema can grab half the data that my API has access to, so it theory it could call both and get all the data. And my GPT has browsing off, so that is the only way it is getting the date. I have to do a little more investigation.
It’s less than 32K tokens for sure, so this may be a good option.
With time, I thought each time the GPT queries the API, it can pass the last time it queried, and my API can do the math and send the data as necessary.
(1): Yep that’s different, a GPT can call any number of schema-defined routes for a single message, and it’ll do it without making it clear to the user. In the GPT Editor you can look at the debug logs to see what calls it’s making. I’d say it’s really likely your GPT is making both calls.
(3): Sure, I’m just saying that you can’t prompt the GPT to only make the call every n minutes. You can prompt the GPT to make the call on every message from the user and then do the math on your backend. It should work most of the time, obviously it won’t be guaranteed because GPTs are probabilistic.
So how the heck does the GPT decide how or when to use an Action? I’ve been grappling with getting an action to run with every message recently and I feel like there has to be a reliable solution to directing a GPT toward a given action for certain circumstances.
How? I’ve gotten it to mostly run at every message, but I can’t get it to actually call the action reliably. It just shows me an “Unknown plugin” most of the time. Frustrating