Token Optimization Question


I’m sending requests to an Azure OpenAI GPT-4-32k instance. I want to optimize the prompt token counts by removing dead spaces; maybe minify prompts, etc.

When I paste a raw JSON prompt into the OpenAI Tokenizer, I see ~17.5k tokens:

When I paste in a semi-processed JSON prompt ( replace(‘’, ‘<no-space’), I see ~5.5k tokens:

<can’t paste the image here due to new member restrictions>

Pretty substantial diff.

However, when I submit the raw JSON prompt to my Azure OpenAI resource, I see the token usage come back as ~5.5k.

So the question: Does Azure OpenAI optimize the prompt? How is the 17.5k token prompt being reduced to ~5.5k tokens?

Welcome to the community @seadude

When you copy-paste from code editor(assuming it’s formatted), it copies the formatting tabs or spaces, used for indentation, as well.

When you pass directly to the API (likely stringifying json before passing) it sends it as a string with the indentation removed.

Hence, it’s not the API that’s doing the work, it’s your code stringifying the json.

Aha! Ok, thank you. That makes sense.

Postman’s code in this case!

1 Like