I’m testing the new release of gpt-4-turbo-preview in an agent using langchain with access to the internet via the web-browser and serpAPI tools.
With the new updates to the model it is outright refusing to return final answers after it uses the web-browser and search tools. Consistently repeating:
“I am unable to provide real-time or updated information from external sources or the internet, including conducting searches or accessing current data.”
Its not doing this in either gpt-4 or gpt-4-1106-preview.
I’ve tried prompting around it but it continues to refuse to provide anything that it considers ‘real-time’ information.
Our initial prompt is about 1500 tokens.
gtp-4-turbo-preview seems faster but is a significant decrease in accuracy of response from gpt-4-1106-preview
It sounds like it does not know that it can get the ‘realtime’ information through a function call somehow? How in this langchain setup does it get the information it needs?
Seems to be the case.
Langchain will output the list of tools and a description into the prompt. This is what it outputs for serpAPI tool : “search: a search engine. useful for when you need to answer questions about current events. input should be a search query”
I’ve also tried adding additional details in the prompt like “you have access to real-time and up to date information by using either the search or webbrowser tools” to get it to recognize that it has access.
Whats weird is that it will use the tools and gather internet data but when its time to output the final result thats when it gets confused and says it cant do it.
Again this is happening specifically in gpt-4-turbo-preview.
gpt-4-1106-preview can be spotty but it will do it, and gpt-4 rarely has issues; all else being equal.
Well I don’t know if you plugin to Assistants or Completions - but for Assistants you’d have to make sure that the functions are part of the Asssitant configuration and your app will need to be able to fullfil the request.
This would be for completions specifically