For example, based on OpenAI API. the word “experimenting” contains 3 tokens. If I hit the max_token limit at “exper”, will it stop at the word before it in the completions response or will it output “exper” and then stop?
Will setting the max_token limit ever result in only generating a word partially?
It’ll stop mid-word … based on settings.
Hope this helps.
It will do the above, not based on word boundaries but tokens.