I created a fine-tuned model and now I’m trying to use it as a classifier. I have 6 possible outputs but they all have different lengths varying from 1 token to 4-5 tokens. I am sending the various inputs with max_tokens set to 5 because that’s the length of the longest class. When the classification result is 1 token, it returns extra stuff appended to the first token, but so far the first token results in the correct answer. I’m trying to understand if there’s a better way to do this.
Is there any way I can get an exact class back instead of a string with multiple classes for the cases when the result is less than max_tokens?
Are you using a stop-word / stop-sequence? Adding one might be a few extra tokens, but it should give you more control.
A ###
BB ###
etc… then you just tell the API when you want it to stop ie “###”…
for fine-tuned, you need to have set-up the dataset that way, though.
@PaulBellow Yes I have fine tuned using a stop word. How do I tell the API to stop when it reaches the stop word?
Thank you very much @PaulBellow - that worked for me.
1 Like
No problem. Feel free to mark my post as “solution” to hopefully help others in the far future!
Hope you stick around.
1 Like
