Logit bias does not work on gpt-3.5-turbo-instruct

API
,
1

As the title says… it doesn’t work.

I am trying to filter out the words “popular” and “popularity”. Even with -100 logit bias, they still appear very regularly. I’ve also tried to filter out words like “Welcome”. But it has no effect.

Array
(
    [prompt] => My prompt here.
    [max_tokens] => 200
    [temperature] => 0.8
    [top_p] => 1
    [presence_penalty] => 0.9
    [frequency_penalty] => 0.8
    [best_of] => 1
    [stream] => 
	[logit_bias] => Array
			(
				[401] => -2
				[785] => -2
				[3740] => -1
				[5450] => -1
				[2638] => -1
				[4023] => -1
				[2968] => -100
				[47568] => -100
				[11533] => -100
			)
)

I am of course aware of how OpenAI treats words that have spaces before, and otherwise. I’ve used all of the ID’s available for these particular words.

Anyone else noticed this? Or has it been confirmed that it doesn’t work?

2

You seem to be using the wrong tokenizer.
The model uses cl100k_base encoding.

image

44348, 
5526,
59552, 
32495, 
8539, 30079, 
23354, 
11859, 30079,
23354
3

I didn’t even know that, so OpenAI doesn’t even have a functional tokenizer for their own produce now? Is there a json encoded version of all of the tokens somewhere?

4

You can use tiktoken as a tokenizer, you can see it implemented here

1 Like