Is it possible to register multiple words for the logit_bias parameter?

API
1

Hello.

Anyone know anything about the title?

First, I checked the token ID of logit_bias with the following code in Python.

encoding: Encoding = tiktoken.encoding_for_model("gpt-3.5-turbo-16k")
token_ids0 = encoding.encode("晴れ")
token_ids1 = encoding.encode("曇り")

And then I got below two token ids as below.

"晴れ" = [45114, 112, 33121]
"曇り" = [27552, 229, 31431]

In addition, I used 3 json formats as below and sent to the API (gpt-3.5-turbo-16k), but different json formats returned errors.

#pattern 1
input_logit_bias = {	
        [	
            "45114": -100,	
            "112": -100,	
            "33121": -100	
        ],	
        [	
            "27552": -100,	
            "229": -100,	
            "31431": -100	
        ],	
}

#pattern 2
input_logit_bias = [	
        {	
            "45114": -100,	
            "112": -100,	
            "33121": -100	
        },	
        {	
            "27552": -100,	
            "229": -100,	
            "31431": -100	
        },	
]	

#pattern 3
input_logit_bias = {	
        {	
            "45114": -100,	
            "112": -100,	
            "33121": -100	
        },	
        {	
            "27552": -100,	
            "229": -100,	
            "31431": -100	
        }	
}

I would appreciate it if someone could tell me who was able to set multiple words in logit_bias.

Thank you.

2

Hi, I’m looking for the exact same thing, to combine words in logit-bias to exclude some pre-made group of words like “Of course” etc. Any help would be appreciate.

Thank you

1 Like
3

The correct syntax is logit_bias={2435:-100, 640:-100},
(in python)

1 Like
4

Thank you for your reply, but theses tokens are not “grouped”, so I mean GPT exclude 2435 and 640, but not 2435 directly followed by 640. Imagine you like to exclude a whole expression like “Of course”, the tokens ID are: 2173 and 3388, if I do {2173 :-100, 3388:-100} it will exclude “Of” and “course” but not exactly “Of course”

1 Like
5

I am not currently aware of any method for specifying multiple logit’s in a specific order.

2 Likes