The response speed of the gpt-3.5 turbo api is very slow. Is there any way to improve faster?
All required times were measured, and it was confirmed that 99% of the time was waiting for API responses.
model = gpt-3.5 turbo
top_p = 0
temperature = 0
average response time = 24.21 sec
200 tests
shortest response speed = 2.5 sec
roll = user " Just tell me 3 types of fruit"
longest response time = 1min 50sec
roll = user “tell me about online advertising”