Hello,
I was testing the capacities of the new model in terms of generate, edit and edit with mask parameters which is awesome.
The response has 3 keys: created, data and usage.
In usage, I’ve seen input_tokens and the output_tokens from image.
I used a simple prompt like ‘A Studio Ghibli-style forest with glowing fireflies at dusk’
The input_tokens were 19 (All in text_tokens)
Then I’ve got all the tokenizers from tiktoken.registry but none of them is 19, they are 13 (for cl100k_base and o200k_base) and 14 (for the others).
Is there a new tokenizer for this model, or maybe internally they add some few words to the prompt?