@mattholdsme and @pconstantine, on average, how many prompt_tokens are you providing, how many completion_tokens are you getting and what is the response_s? These are available on the response object returned by the Chat Completion API.
The issue for me has since been restored. I noticed improvement around December as I previously predicted. The response times are now back to being as expected.
hey @mattholdsme do you have numbers to share? I’m having trouble getting value from these posts:
I am on tier 4 and using gpt 4 1106. The response times for this model seem reasonable. Still a bit sluggish
This part seems both vague and ambiguous IMHO.
The response times are now back to being as expected.
How many completion tokens are you getting and how long is it taking on gpt-4-1106-preview? If you can’t measure, pls provide some approximate numbers – e.g. "I get about 10/50/500 words in less than 10s / 30s / 1min…
Best regards!