Hello. I am using API to get product descriptions. Each request takes more than 30 seconds. How can you speed up the response time?
model => gpt-3.5-turbo
promt => ~600-700 chars
last response time => 47sec
From my tests, input tokens number doesn’t have any weight on this, only completions token. So perhaps you could simplify the question or enforce shorter answer, so it spends less time completing text. I also believe you have some network problem, perhaps try some VPN.