Hey curious to know if there is a public benchmark of OpenAI api in terms of speed of different models with some variables like time, model, parameters …
Maybe something like https://status.openai.com/ but for speed
Eventually a comparison with other AI APIs?
PS: would be happy to benchmark the API if given free credits of course 
Not mine, but https://www.gptstat.us
@taivo wrote a nice blog post about his experiments with speed. His conclusion was that Azure was almost twice as fast. Which makes sense since they have more hardware.
Looks like he wrote another post too, with some speed tips.
https://www.taivo.ai/__making-gpt-api-responses-faster/
Anyone know a comparison of OpenAI’s LLMs token/s vs open source LLMs through vLLM for example?
You cannot compare the token generation rate of open source models to OpenAI API models.
The first is hardware-dependent, and the second is secret hardware and payment tier-based.