How are you handling function calling in production.
Are you directly using OpenAI function/tool calling for the workflow itself, or mainly using the LLM for intent recognition + parameter extraction, while handling the actual flow and execution logic internally?
Especially for multi-step agents and production systems.
For production workflows, I’d usually let the LLM handle intent and parameter extraction, then keep state, retries, permissions, and side effects in deterministic orchestration. Tool calling is still a good contract, but I wouldn’t make the model the workflow engine for anything business-critical
They are for different purposes.
What are you trying to achieve? Describe the problem you are trying to solve in more detail?
Function calling is most appropriate when you are trying to delegate decision making and action taking to the LLM. A good example of that is a chatbot where you are providing the bot with tools to answer unpredictable queries. Another great example of that is a coding agent.
On the other hand you might have a more deterministic workflow that needs to make more discrete use of LLM properties, for example summarising text in batches.
I was writing an article on function calling, and gathering insights from developers on how they use it in production environment. One of the major challenges we have seen with customers is:
The way we solve is by creating workflows that has multiple api call/functions within it, and use OpenAI to figure out the workflow to trigger.
Curious to know how you use function calling for such use case?
Yeah that sounds reasonable.
LLM choice of which function call to invoke can degrade after you give it more than a few especially if some functions are too similar to others.
If you can gather things in predetermined sequences that sounds a great solution so long as it’s obvious to the LLM which of those workflows to initiate.
yes it becomes tricky and accuracy degrades once there are many functions. Thanks for the input @merefield
Make sense, thanks for the input