I’ve just published a video based on a real text-analysis workflow involving several thousand API requests and millions of tokens per trial.
The main idea is simple: the model should do only the work that actually requires a model.
I compare four trials and look at how cost and behavior change when adjusting:
- input caching
- output token generation
- task decomposition
- function calling
- structured responses
- context cleanup
- model selection per step
- reuse of previous results to reduce decision space
The workflow is orchestrated by the application rather than by an autonomous agent, and each model is used for a small, well-defined task.
The experiments also produced some useful failures: repeated function-call loops, overclassification, poor multi-word term extraction, and cases where a small prompt or schema design decision had a surprisingly large effect on token usage.
The video includes usage and cost data, per-step analysis, recorded contexts, and some preliminary observations about result quality.
Full video: https://youtu.be/X3RT8wnQQRE
I’d be very interested in comparing approaches with anyone working on similar multi-step API workflows.