I did everything to remove the word ‘assist’, yet it’s not registered correctly, somehow…
Have you tried a system message saying "Greet the user by saying “Hello, how can I help you today?”
That would work, but i specifically want to ban some certain words, starting with this…
I get this
Yeah, the gpt-3.5 uses cl100k_base
so the tokens are correct, it’s a big mystery to me if the API is not actually registering it OR the “assist” is made up of smaller tokens; ass-ist | as-s-ist | etc. I cant use ‘logprobs’ return either cuz they banned it in chatgpt. So really stuck here 
Are you saying the OpenAI tokenizer page is wrong? 
That has some interesting implications if so…
that page is the old tokenizer, for < gpt3.5
Did not realise that! you live and learn.