I am making API calls to OpenAI and it takes about 25 seconds to respond. If there is any technique to reduce this time. Specifically, Are Enterprise APIs faster than normal ones?
Model: gpt-4o-2024-08-06
Response_Format: Structured Json
Avg Output tokens: 1000 or less
(Input prompt contains one image also)
Back on Azure, I noticed that the newer API version for structured outputs was significantly slower. I wasn’t even using structured outputs as the response type - I just thought I should try the latest version. I’m not sure how that relates to OpenAI(.com), though.
It might be an interesting exercise to test your current call alongside an older API version leveraging something like Instructor to extract the same structured JSON model.