Feature request: Query token counts via API

[sps]

Hi @gg

Here’s something:

Token counter (Codex) for a back-end service

After some searching, I found that @simonl from the community has already built a tokenizer that works for both GPT-3 and codex. Here’s the GitHub Here’s the original post: Codex Tokenizer Logic - #2 by simonl