Why Batch API over synchronous APIs

API
1

I’m not able to understand that why using the already existing synchronous APIs for large volumes of requests which lie within the rate limit is not a good idea and why Batch API should be preferred there . Cost is not a concern so the 50% deduction is not a big attraction .

2

If cost is not a concern, then use whatever you want.

The entire point of the batch API is to trade speed for cost savings, if you didn’t need that then don’t use it.

¯⁠\⁠_⁠(⁠ツ⁠)⁠_⁠/⁠¯

3

But is there any advantage that for large volumes asynchronous processing might be better and real time processing for large volumes reduces efficiency ? Anything like that ?

4

Advantage for who?

5

Advantage for the users of Batch API .
Actually I need to give advantages of batch API . I want to say that how it is always better for large volumes of data compared to synchronous APIs. But I’m not able to understand the main issue with using synchronous APIs for large volumes of data.

6

The advantage to the user is the cost savings.

1 Like
7

Some big disadvantages are:

  • the current polling based solution. We really need pub sub to be added for push based job completion readiness notification.
  • the requirement to create a completely separate set of code to manage batch interaction

on top of the obvious time trade-off.

8

Batch API is useful when the goal is to cut costs while asynchronously processing large amounts of data. Example use cases include, but are not limited to, digitizing/transcribing written material, embedding large sets of data, and evaluating large amounts of data.

Another benefit is that when you have large amounts of data, processing takes time. This can help you avoid blocking your infrastructure for data so large that your current tier could be hitting Requests per minute and/or Tokens per minute limits.

Compared to using standard endpoints directly, Batch API has:

  1. Better cost efficiency: 50% cost discount compared to synchronous APIs
  2. Higher rate limits: Substantially more headroom compared to the synchronous APIs
  3. Fast completion times: Each batch completes within 24 hours (and often more quickly)