Hangs with batch async API calls using asyncio.gather, even with StrictLimiter

API
1

I run a lot of batch API calls using asyncio.gather() similar to the example below. I use asynciolimiter.StrictLimiter to limit the rate of API calls. However, I find that some of the calls just hang and take forever to complete. Is there a reason for this? Am I hitting some API limit? How could I prevent this? I also set the max_tokens to prevent the output from getting too long. Any insight would be appreciated.
Thanks!

import asyncio
from openai import AsyncOpenAI

aclient = AsyncOpenAI()

async def process_prompt(prompt):
    response = await aclient.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}],
    )
    return response.choices[0].message.content

async def main():
    prompts = ["Explain quantum entanglement", "Summarize WW2", "Define monad"]
    tasks = [process_prompt(prompt) for prompt in prompts]
    results = await asyncio.gather(*tasks)
    print(results)

asyncio.run(main())
2

Looks like one solution is to use asyncio.wait_for on the completions.create call still not ideal, but it works.