OpenAI API: Platform/tokenizer giving very incorrect/inconsistent counts?

I was using the token-counter earlier today (about 3h-4h ago, and I felt the results were quite wrong.)
I had counted the tokens for the same string before, and they were close to 1500. Counted during/after shortage, same string = 2700.
Counted again right now and they are back to 1500-level.