Is it possible to reduce ChatGPT API response time?

ChatGPT
1

The ChatGPT API response time seems to be around 15 to 20 seconds.

Is there a way to reduce the response time? Would having a ChatGPT Plus subscription help?

1 Like
2

  1. Reduce the number of network hops between you and the API.

  2. Increase your network service quality.

No.

:slight_smile:

1 Like
3

It is dependant of the traffic as well. For the same generation, I have encountered a broad range of response times based on the time of the day.

1 Like
4

are you using curl for req? Or what protocol do you use to access the API?

5

If you are not using streaming (stream=true), you could turn it on to make it look like things are happening faster.

It won’t actually make it quicker, but the user will see things happening on the screen 9assuming you post the progress as it comes in)

Streaming is quite a bit harder to code if you are not using a library.

2 Likes
6

@nguyenanhdon.qn I am using the Python module as described in Introducing ChatGPT and Whisper APIs.

import openai

completion = openai.ChatCompletion.create(
  model="gpt-3.5-turbo", 
  messages=[{"role": "user", "content": "Tell the world about the ChatGPT API in the style of a pirate."}]
)

print(completion)
1 Like