Hello!
I’ve implemented simple FSM approach to the ReAct pattern. Idea is following:
I’ve got 3 states - idle, waitingForLLM, waitingForTool
when in idle I’m sending message to LLM and changing state to waitingForLLM
when in waitingForLLM state if receive from LLM reasoning type of message I’m sending it back to the completion endpoint(with the whole history) and stay in the same state
when I receive assistant type of message I send the response to the user and change state to the idle
when I receive tool call I ask for the tool to run and change state to waitingForTool
when I receive response from tool I send it to the LLM and change state to waiting for LLM
OK, so I’ve described this rather simple flow. What are my problems - cant figure out the prompt to always send one message at the time reasoning, assistant(those messages I want to handle with the structured output within the content, I have a simple structure for a json with type and message) OR the tool call in the tool call array in the API object.
With various approaches I’ve experienced following:
- in structured output I’ve got two separate json objects like that:
{"type":"reasoning", "message":"..."}
{"type":"assistant", "message":"..."}
- I get structured output and the tool call in the same response - I try to specify in the system prompt to use only one at the time, but it doesn’t listen.
Models I used for testing are GPT-4o-mini and GPT-4.1-mini.
Can someone give me some pointers how to handle prompting in this case?