Hello community,
I am experiencing a critical consistency issue with an Agent I built using the OpenAI Agent Kit.
The Setup:
-
I have an Agent configured with a specific System Prompt and a set of Tools (RAG retrieval for a product catalog).
-
The goal is to retrieve specific Item UUIDs and Prices from the catalog.
The Issue:
-
In the Agent Kit “Chat” Tab (UI): The agent works perfectly. It correctly calls the retrieval tool, reads the UUID/Price from the tool output, and refuses to answer if data is missing. Zero hallucinations.
-
In my Workflow (API Integration): When I connect the same agent configuration to my workflow, it starts hallucinating. It invents UUIDs that don’t exist in the RAG source and makes up prices.
What I’ve Checked:
-
The System Prompt text is identical in both environments.
-
The RAG tool availability is the same.
My Question:
Are there hidden parameters or “silent” system instructions that the Agent Kit UI applies in the “Chat” tab that are NOT automatically applied when connecting via the API? specifically regarding:
-
Default Temperature settings?
-
How Tool Outputs are fed back into the context window?
-
Implicit “Strictness” controls?
Has anyone else seen this “Dr. Jekyll and Mr. Hyde” behavior between the UI and the API integration?
Thanks!
workflow image for reference:
