Please explain the Tokens per minute metric

[Diet]

It’s both - and it’s complicated

the input tokens are estimated, and added to your max_tokens - so you can think of it as total token throughput per minute of sorts. they’re not actually using tiktoken at that level, it’s more of a flooding prevention sort of thing.

Rate limit reached for 10KTPM-200RPM

The reason why you are encountering that error is because gpt-4 has rather low limits for tokens-per-minute. Tokens are the internal AI encoding that represents words and parts of words as pieces. The rate limit doesn’t actually count the tokens though: it estimates based on characters you input. However, it does consider the value you specify in max_tokens as counting against the rate limit in tokens. If you specify a large max_tokens, you may be blocking yourself even though you are only ge…