Hi All, I’m developing an application that depends on communicating moderate size word documents to Gpt3.5 turbo API and receiving a response , the problem is that processing one request takes about 80 seconds which is huge noting that I intend to make this a corporate grade application that would …

Sometimes the API can be slow due to high demand. You could try using Azure OpenAI, which is approximately x2 faster for half the cost. If you don’t want/can’t use Azure OpenAI, you could try breaking down your requests into smaller bits. For example, instead of moderating the entire document at on…

Thanks a lot ,I thought OPenAI azure would be more expensive

[image] aelfouly: Hi All, I’m developing an application that depends on communicating moderate size word documents to Gpt3.5 turbo API and receiving a response , the problem is that processing one request takes about 80 seconds which is huge noting that I intend to make this a corporate grade a…

Hi aelfouly. You have a good question and one that is only hinted at within your account and documentation. Usage tiers You can view the rate and usage limits for your organization under the limits section of your account settings. As your usage of the OpenAI API and your spend on our API goes up,…

This is really informative , thank you so much

Have a look to one of the community leaders tips on thanking, references provided in the thread. I’ll link it here: [image] Interested in learning prompting skills Prompting A better way to say thanks than to create a reply is to click the heart [image] at the bottom of a …

Thanks , but this talks about the limits not the processing speed , what I’m facing is 80 secs of delay when processing a single request which could be drastic when scaled up to a large number of documents , of course the limits problem will appear then buts not what I’m faacing now , I need to proc…

I did some quick testing using a free-tier account. 3.5-turbo-1106 has decent speed. 3.5-turbo outputs at half speed, which is not as bad as it was in October and November. To diagnose a speed issue, the first thing to do is go to the playground and compare streaming speed with free ChatGPT.

Playground IS your API account and its capabilities. So you get an evaluation similar to if you programmed it yourself. Programming it yourself would let you measure output speed better though, in terms of time to get the first token, and then the token generation rate after that - when using chat …

Slow responce from GPT 3.5 Turbo API

API

maurizio.chiaro December 16, 2023, 7:57pm 7

Have a look to one of the community leaders tips on thanking, references provided in the thread. I’ll link it here:

Topic		Replies	Views
How can I improve response times from the OpenAI API while generating responses based on our knowledge base? API chatgpt , api	3	25530	November 9, 2023
GPT-4 API to slow when you have to work with a 46 second time out API	11	2986	July 30, 2023
Assistants API Performance API api , assistants-api	11	3103	March 21, 2024
Assistant API Performance is Very Slow API plugin-development , api	10	5644	March 7, 2024
Why Assistants API is Slow? Any speed solution? API api-speed , openai , rag , assistants-api	15	9745	September 10, 2024

Slow responce from GPT 3.5 Turbo API

Related topics