In OpenAI’s article titled “Introducing Structured Outputs in the API” there’s an entire section on how they’re using “Constrained decoding”.
Our approach is based on a technique known as constrained sampling or constrained decoding… To do this, we convert the supplied JSON Schema into a context-free grammar (CFG)…
A lot of the verbiage used in the OpenAI release give vibes from this paper: Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generation (arxiv.org)