I don’t know if that is by design. It looks like a design flaw to me.
I think the openapi.yaml should be accessed only once during the plugin installation or during the startup of a new chat. Why would the ChatGPT plugin framework need to access it for every single prompt?
Imagine, if your plugin becomes modestly popular, say 100 concurrent users, and your openapi.yaml is 40k, then your hosting server would have to reserve 4M (40k * 100) bandwidth and CPU power just to serve the openapi.yaml!!
Those who go to the OpenAI developer conference, please, ask the question.