What tokenizer is GPT4.1 one using?

Can I assume that it is using GPT4o’s tokenizer? Because I kind of use tiktoken to roughly estimate my token count and based on what I see, it is slightly underestimating it. Unsure if this is because GPT4.1’s tokenizer is different from GPT4o’s or if that’s normal.

3 Likes