Posting this in case it saves someone else the debugging time it cost me. I’m running gpt-oss-120b (via Groq’s API) for a feature that generates short personalized text based on user history — nothing fancy, just a completion call in a Next.js API route. Every so often, with no pattern I could initially see, the call would come back with an empty completion. No error thrown, no obvious rate limit, just… nothing generated, which downstream meant a broken response to the user.
It took a while to isolate because the failure wasn’t consistent enough to reproduce on demand. What eventually helped: setting reasoning_effort explicitly to low and bumping max_tokens up. My working theory is the model was spending its token budget on internal reasoning before it ever got to the actual output, and with a low enough max_tokens ceiling, it could exhaust the budget before producing anything visible, which reads as an empty generation with no error, since technically nothing went wrong on the API’s end.
Curious if anyone else running gpt-oss-120b for short, latency-sensitive completions has hit this, and whether there’s a more precise way to reason about the token budget split between reasoning and output than what I landed on through trial and error.