Improve response time of GPT

Hi,
We are developing a tutoring platform and are using GPT4 Vision for it.

Due to the complexity of the tasks, the answer takes up to 2 minutes and the users find this super annoying.

To test our prompts we use chat.openai.com as well as typingmind.com

On both platforms, the output is significantly faster and GPT starts writing the message after just a few seconds.

Do you know why that is?
Is there a way to get the answer in parts and start displaying the answer?

Hi!

It’s called streaming :slight_smile:

https://platform.openai.com/docs/api-reference/streaming

1 Like