X-ratelimit-remaining-tokens header missing for batch API

API
1

Hello

I am trying to implement batching for rather large amount of requests. I am very likely to exceed my batch queue limit TPD.
I see that for the completions API we get the x-ratelimit-remaining-tokens header, but I see that the header is missing when I curl the /batches API.

Is there a way to find the x-ratelimit-remaining-tokens header for the Batch Queue either through another API call or some other trick?

It is not a problem for me that it exceeds, ie. I’m not planning on moving up another tier just yet, I just need to be able to pull the remaining tokens header so I can do the error handling on my end.

I’m using openai-php and the BatchResponse has functionality to give the remaining-tokens, but it seems it is not receiving it from the OpenAI API. So I tried using curl to see if the issue was the PHP package, but curl does not receive the header either for /batches and /files endpoints.

Anyone able to enlighten me on the issue :slight_smile:

1 Like
2

I’ve run into the same issues and there doesn’t seem to be any way to get at that information other than either keeping track of your work in progress on a fine-grained level or to have a more complicated submit loop where you initiate a batch and then poll the API until you see that it has finished validation. If it finishes, you move on to submit the next one and if not, you sleep for a while.