I’ve shared a video with results from those experiments here: Exploring a New UX for Multi-Agent AI with the Realtime API
We ended up implementing a simple state-machine that dynamically adds only the tools relevant to each stage of a workflow. This approach has been working really well. While it required quite a bit of scaffolding and guardrails, the strategy of dynamically changing the tools and showing the model as few functions as possible at any given time has proven effective.