We are trying to pre process our data between agent and our API. We have set temperature at .5 but the responses are too big and we are exceeding amount of tokens we can use.
Has anyone run into this and have any tips for preprocessing before calling the model? Our API specs are quite large.