Token Calculator feature available as API

General API discussion
1

Hi All,

I want to know how many tokens does OpenAI count for a given text when using OpenAI models/APIs. In other words, I want the feature to calculate number of tokens for a given text just as tokenizer(OpenAI API) does but using an API.

Thanks

2

I guess, there’s an npm package for that if you make a search

3

More specifically this one

The tokens are stored locally, thus fetching a decode/encode is immediate.