Managing High Costs in Large-Scale JSON Processing

API
,
1

I have a function in my assistant. This function, consisting of a massive 2000+ line JSON schema, processes user queries to construct JSON requests for an external API. It has been performing well but the costs are insane, around 15 cents per run.

Has anyone encountered similar issues with large-scale JSON processing, and how did you manage the costs?

2

You can fine-tune a model or call a dummy function which then you can send GPT the full schema afterwards to prevent it being used for every message