40

I think the idea that streaming would help you to avoid timeout of 30 sec running your server script is wrong. Although you start sending the data sooner, the timeout will hit you after 30 seconds and the data that have not been generated by the API will be lost.

1 Like
41

Try it out and let us know how it goes.

1 Like
42

Yeah, I am getting a 503 error. And Chatgpt free version have no issue here in Bangladesh.

2 Likes
43

I am having delays and errors as well…while losing money basically.

1 Like
44

We are getting 503 Errors in Goa India since 10 am this morning.

1 Like
45

Facing same issue since last couple of days. It has taken the delay times up to few minutes from few seconds. Using it in India with turbo-3.5 model. Not something we expect from openAI.

1 Like
46

I’m having same issues. API got so unreliable and slow that any tools based on it are useless at the moment.

1 Like
47

I solved the issue after reading one of comments. I bought $50 credits with OPENAI company account, and the problem was solved, and the queries were faster, more reliable and with a better Chatgpt model. Crazy !

1 Like
48

Buy $50 credit, this is how I solved my issue.

1 Like
49

I bought $50 credits with OPENAI company account, and the problem was solved.

1 Like
50

I bought $50 credits with OPENAI company account, and no issues anymore.

1 Like
51

Please do not add on to this topic unless you have tried the solution:

2 Likes
52

Further to my earlier posts yesterday, this seems to have come right and is back to normal for me for the past several hours. I did not add credit nor change any billing/plan. HTH.

1 Like
53

I’ve tried the solution, and it works.

2 Likes
54

Awesome. However, it’s still unreliable for my use case with my regular api key.

1 Like
55

Imagine running a chain and each call taking minutes! It’s basically not working anymore

56

This solution worked for me. Thank you

57

In my pipeline all the gpt-3.5-turbo calls seem to time out since Nov 1st… gpt4 works fine.

According to OpenAI Platform I am at usage tier 3, so the credit buy will probably not fit fix it for me.

gpt-3.5-turbo 160,000 TPM 5,000 RPM
gpt-4 40,000 TPM 5,000 RPM

This is quite urgent. Customers already complaining

58

Looking at the payment requirements adding credit to get you to tier 4 should help.

image
image797×698 32.6 KB

1 Like
59

In your backend, you should monitor streams, close connection and retry quickly if you don’t get a response for the first token within 2 seconds. This will maintain “satisfaction” instead of having users wait for a response that will never come.

If it is about overall rate, a token rate generation test you can run:

https://community.openai.com/t/gpt-3-5-turbo-api-response-is-slow/424046/10

My “tier 3” has performance similar to two weeks ago, testing three hours before the biggest daily slowdown.

For 2 trials of gpt-3.5-turbo-0613 @ 2023-11-02 05:08AM:

Stat Minimum Maximum Average
latency (s) Min: 0.414 Max: 0.4971 Avg: 0.456
total response (s) Min: 13.8798 Max: 14.4488 Avg: 14.164
total rate Min: 35.435 Max: 36.888 Avg: 36.162
stream rate Min: 36.6 Max: 37.9 Avg: 37.250
response tokens Min: 512 Max: 512 Avg: 512.000

For 4 trials of gpt-3.5-turbo-0301 @ 2023-11-02 05:08AM:

Stat Minimum Maximum Average
latency (s) Min: 0.359 Max: 1.3942 Avg: 0.638
total response (s) Min: 7.4474 Max: 9.0956 Avg: 7.936
total rate Min: 56.291 Max: 68.749 Avg: 64.942
stream rate Min: 66.4 Max: 72.5 Avg: 70.100
response tokens Min: 512 Max: 512 Avg: 512.000

However, the gpt-3.5-turbo output has degraded to stupidity and now mocks you for attempting to get a desired length.

AI output

Prompt: Write an article about kittens, 80 paragraphs

Terrible output, producing numbered sentences as “paragraphs” and stopping after number 21:

Kittens are undoubtedly one of the most adorable creatures on Earth. With their tiny
paws, fluffy fur, and playful nature, it’s no wonder they capture the hearts
of millions. In this article, we will explore the world of kittens, their
behavior, care, and the joy they bring to our lives.

  1. Kittens are baby cats, typically born in litters of four to six.
  2. They are born blind and deaf, relying solely on their sense of touch
    and smell.
  3. As they grow, their eyes open around 7-10 days, revealing their mesmerizing
    colors.
  4. Kittens start exploring their surroundings at around three weeks old,
    stumbling and wobbling as they learn to walk.
  5. Their playful nature emerges as they engage in mock battles with their
    siblings, honing their hunting skills.
  6. Kittens are incredibly curious creatures, always investigating every nook
    and cranny they can find.
  7. They love to chase anything that moves, from a simple piece of string
    to a laser pointer.
  8. Kittens are known for their acrobatic abilities, effortlessly leaping
    and climbing to reach new heights.
  9. Their tiny meows are irresistibly cute and can melt even the coldest of
    hearts.
  10. Kittens are highly social animals and thrive on companionship, both
    with humans and other animals.
  11. They are quick learners and can be easily trained to use a litter box
    and scratching post.
  12. Kittens have sharp little teeth and claws, which they use during playtime.
    It’s important to redirect their attention to appropriate toys to avoid
    accidental scratches.
  13. Their fur is incredibly soft and requires regular grooming to keep it
    clean and tangle-free.
  14. Kittens are natural-born hunters, and their instinct to pounce and
    chase is ingrained in their DNA.
  15. They have an innate ability to sense movement, making them excellent
    at catching bugs and small rodents.
  16. Kittens are known for their love of napping, often curling up in the
    most adorable positions.
  17. They are highly adaptable and can adjust to various living
    environments, whether it’s a small apartment or a spacious house.
  18. Kittens are notorious for their mischievous behavior, often getting
    into things they shouldn’t.
  19. They are fascinated by water and may try to play with it, leading to
    some hilarious and wet encounters.
  20. Kittens are incredibly resilient and can bounce back from minor
    accidents or falls.
  21. They are natural-born climbers and mayAs one of the most beloved
    animals in the world, kittens are a source of joy and happiness for many
    people. These adorable little creatures are known for their playful nature,
    cute appearance, and affectionate personalities. Whether you are a cat lover
    or not, it is hard to resist the charm of a kitten.