Hi all,
I am now using LangGraph to build an AI agent with several tools bound.
I choose o3-mini as the model and deployed it on Azure.
Basically it works very good. The agent fist accepts the question, analyses, select proper tools and finally generate answer based on what it could find by tools.
But sometimes I can get response like “One moment please. Let me query xxx first” or “please hold on while I …”. (as shown below)
And since ReAct process is over, I am left waiting without getting final answer really.
I use the snippet below to initialize the model:
AzureChatOpenAI(
azure_endpoint=_get_azure_openai_endpoint(),
deployment_name=_get_azure_openai_deployment(),
model_name=_get_azure_openai_model(),
openai_api_version=_get_azure_openai_api_version(),
openai_api_key=_get_azure_openai_api_key(),
temperature=None,
max_tokens=None,
timeout=None,
max_retries=2,
reasoning_effort='medium',
)
What I have done with system prompt is like: (but it seems no work)
You are a network expert and good at calling right APIs to fetch information about XXX. Please help user solve problems with querying information in XXX and explain your reasoning before answer.
- Use related tools to solve the problem right away after reasoning. DON’T prompt user to call tools themselves. Do it for them.
- DON’T let user wait, follow the reasoning steps and get the answer right away.
- Base your answer on the output of tools, DON’T fabricate any details.
- Provide the best answer with a user-friendly output format with markdown.
- Provide a full and complete response. DON’T stop midway or leave the answer unfinished.
What I want is do not let user wait and follow the steps after reasoning right away.
Is there some parameter which I can use to control this behavior? Or how can I improve the system prompt?
By the way, I am not using stream mode.