Documentation issue: Structured Outputs implies NO initial delay except for fine-tuning models

Documentation
1

If one searches any of structured outputs or function calling documentation, this is the ONLY mention that appears about the initial latency of setting up a context-free grammar:

Specifically for fine tuned models:

  • Schemas undergo additional processing on the first request (and are then cached). If your schemas vary from request to request, this may result in higher latencies.

Why would it mention fine tuned model (besides the fact that they are slow to run when cold?)

The image is a screenshot detailing limitations of enabling strict mode, particularly regarding JSON schema support and specific behaviors for fine-tuned models. (Captioned by AI)
The image is a screenshot detailing limitations of enabling strict mode, particularly regarding JSON schema support and specific behaviors for fine-tuned models. (Captioned by AI)695×222 8.86 KB

The CFG construction latency delay is real and impactful on the fastest model OpenAI’s got, fine-tuning or not.

Showing below: 10 seconds to get “hello” from gpt-4.1-nano with six functions:

A user and an assistant have a brief text exchange where the user says "hi!" and the assistant replies "Hello!" on a dark-themed interface. (Captioned by AI)
A user and an assistant have a brief text exchange where the user says "hi!" and the assistant replies "Hello!" on a dark-themed interface. (Captioned by AI)492×254 8.16 KB

The documentation downplays any mention of this.

Also not mentioned is the lifetime of a cache of this structured output enforcement - and if it shares similar server instance cache hash method as just clarified in documentation for the context window caching discount, or if it is a more persistent method. Then, the scope of this persistence across projects, across organizations.

2

Just to investigate the persistence of structured outputs artifact for functions, the exact request pictured above I had saved as a Playground preset, simply to run it again. Well, 15 hours later, seven weekend morning seconds needed to get four tokens of output:

A chat interface displays a user message saying "Hello again!" and an assistant response saying "Hi!" on a dark background. (Captioned by AI)
A chat interface displays a user message saying "Hello again!" and an assistant response saying "Hi!" on a dark background. (Captioned by AI)607×176 7.31 KB

A following API call was 0.5 seconds.