Hi everyone,
I’m using structured output to obtain valid JSON responses. According to the documentation, the first time a schema is used, OpenAI converts it into a CFG (context-free grammar), which incurs a latency of around 10-60 ms.
Does anyone know how long OpenAI retains the CFG?