It’s a token by token basis (parts of words), which is part of the problem with counting. But yeah, not being able to “count” as it’s generating also makes it difficult.
There’s a new long-output model coming out, I believe, so OpenAI is aware of the limitations and working on them.