is the primary way to simply detect repeat text/tokens in the stream? (then abort the request/stream)
do you still pay for all the tokens? even when aborting the request?
thanks
I believe you still pay for the tokens, but I could be wrong.
Sounds like your fine-tune model is overfitting or the temperature might be too low. Can you share more on your settings and what you’re trying to achieve?