I’m using the web search tool with GPT 4.1 via the responses API.
Before this week, the web search tool was billed as the “4o” version that does not incur additional cost for search tokens.
Starting a few days ago, that web search tool is listed in the API Playground as the “Deprecated” version. There is now a “Latest” version that has an additional feature of allowing me to add a list of whitelisted websites for the search tool to consider. Notably this “Latest” version does charge for additional search tokens, which is fine.
The issue is that this “Latest” web search tool version totally ignores the system prompt. I could have a system prompt that says “output nothing” or really anything, and the model will return the search results for the user query in whatever format it wants.
Is this a bug? I can’t imagine ignoring the system prompt is intended. Is there any documentation or announcement for this new version? The “Latest” version just showed up one day but I can’t find anything about it.
The same symptom is not new: an injected system message that will only produce “web search results” in the exact format OpenAI wants.
The web search tool cannot be used as RAG to improve AI knowledge. It can only give the seen style, an AI-looking “search results” - snippets with links to the originating web site.
The best you can do with OpenAI services is use Chat Completions’ search model, which also only does that (but at least you know what to expect with one price and one call) - and use its output as your own function return.
When I use the “Deprecated” web search tool, I can use the system prompt to very effectively shape how the model interprets and formats the output of the search result.
When I use the “Latest” web search tool, I cannot do this or have any effect on the model whatsoever.
It appears that with the “Latest” search tool I can inject my intended system prompt into the user prompt as a workaround (which works), but that’s a little silly.
You can try “injection” as an immediate developer message also, to have a bit more authority. See how that goes.
However, you can’t know if the web search tool will actually be seen as useful for a question. So, your tune-up may affect the answering of a non-search-answerable input if you have developer+user or are modifying the user’s chat.
The thing could be completely a different model run on the input context after being triggered, like the image tool is completely taking over and running the context with a different model, and the AI-sent prompt means little. I like transparency and not undocumented uncontrollable billed behaviors.
Thanks for the suggestions. I’m hoping someone from OpenAI can take a look at this and confirm that this is intended or a bug. It doesn’t make sense that the system prompt has no impact all of a sudden.
Why not give gpt-5-mini (via the response API) a shot?
You can use the “instructions” parameter as your system prompt - I use it to give output formatting instructions (among other things). It works quite well.
Of course, you can change the highlighted parms.
{“model”: “gpt-5-mini”,
“tools”: [{“type”: “web_search”, “search_context_size”: “high”, “user_location”: {“type”: “approximate”, “country”: “US”}}],
“reasoning”: {“effort”: “medium”},
“text”: {“verbosity”: “medium”},
“instructions”: “Put your system prompt here”,
“input”: “Put your question here”
}