Classification question - what should max_tokens be?

API
, ,
1

I created a fine-tuned model and now I’m trying to use it as a classifier. I have 6 possible outputs but they all have different lengths varying from 1 token to 4-5 tokens. I am sending the various inputs with max_tokens set to 5 because that’s the length of the longest class. When the classification result is 1 token, it returns extra stuff appended to the first token, but so far the first token results in the correct answer. I’m trying to understand if there’s a better way to do this.

Is there any way I can get an exact class back instead of a string with multiple classes for the cases when the result is less than max_tokens?

2

Are you using a stop-word / stop-sequence? Adding one might be a few extra tokens, but it should give you more control.

A ###
BB ###

etc… then you just tell the API when you want it to stop ie “###”…

for fine-tuned, you need to have set-up the dataset that way, though.

3

@PaulBellow Yes I have fine tuned using a stop word. How do I tell the API to stop when it reaches the stop word?

4

Here you go…

https://help.openai.com/en/articles/5072263-how-do-i-use-stop-sequences

5

Thank you very much @PaulBellow - that worked for me.

1 Like
6

No problem. Feel free to mark my post as “solution” to hopefully help others in the far future!

Hope you stick around.

1 Like