Batch API is useful when the goal is to cut costs while asynchronously processing large amounts of data. Example use cases include, but are not limited to, digitizing/transcribing written material, embedding large sets of data, and evaluating large amounts of data.
Another benefit is that when you have large amounts of data, processing takes time. This can help you avoid blocking your infrastructure for data so large that your current tier could be hitting Requests per minute and/or Tokens per minute limits.
Compared to using standard endpoints directly, Batch API has:
- Better cost efficiency: 50% cost discount compared to synchronous APIs
- Higher rate limits: Substantially more headroom compared to the synchronous APIs
- Fast completion times: Each batch completes within 24 hours (and often more quickly)