Chat Completion API service_tier parameter

Does anyone want to elaborate on this new field in the completions API? How to subscribe to the scale tier service and what are the benefits?

string or null

Defaults to null

Specifies the latency tier to use for processing the request. This parameter is relevant for customers subscribed to the scale tier service:

* If set to 'auto', the system will utilize scale tier credits until they are exhausted.
* If set to 'default', the request will be processed in the shared cluster.

When this parameter is set, the response body will include the `service_tier` utilized.

If you are an Enterprise partner with dedicated compute units or provisioned services, you probably know more about this, while the way enterprise and partners services work as far as services and billings is completely opaque from the outside.

You can read other provider’s billing based more closely on the dedicated hardware, such as Azure AI search or Google, and imagine what’s going on with the parameters provided in the API (which I first noted when committed a few days ago). Specifying machine types or scale tiers  |  AI Platform Training  |  Google Cloud

OpenAI committed a bit more to the API documents:

This parameter is relevant for customers subscribed to the scale tier service

Scale tier has been stealthfully published hours ago, in a speculative document, but how one qualifies and subscribes is by contacting your existing “account director”:

With Scale Tier, you can purchase input and output token units. Each input unit costs $3,800 and entitles you to 20k input tokens/min. Each output unit costs $1,200 and entitles you to 2k output tokens/min.

This appears to be for gpt-4o and future models. The period of “subscription” is a month, with yearly commitments as an option.