Problem
When calling the OpenAI GPT-4.1 model for LLM conversations, our system occasionally encounters long delays when receiving responses, resulting in triggering timeout errors.
After investigation, we suspect that internal client errors may have occurred within the model. Although we implemented Try Match exception handling at the request level, the model did not return the corresponding error code and did not return it for a long time
API Version: openai 4.1 2025-04-14 Model Region: east-us Applicable Region: Asia-Pacific Frequency: The issue occurs sporadically in approximately 1% of requests, with no clear correlation to request content identified so far.
Troubleshooting: Confirmed that API keys and quotas are normal. Basic simple request tests show no abnormalities. Completion times for other requests’ first and last nodes during the same period show no anomalies.
Harden the client: implement bounded retries with exponential backoff + jitter for network/5xx/429 classes, and treat timeouts as retryable. (See official error-code guidance.)
Tune timeouts intentionally: the official SDK default timeout is 10 minutes; increase it for long generations or switch to streaming to reduce “silent waiting.”
Check concurrency bursts: even if RPM is fine, short concurrency spikes can produce tail-latency; throttle/queue and add circuit breakers on your side.
If you post one anonymized example (timestamp, region, model, x-request-id, your X-Client-Request-Id, request size, streaming on/off), others can help pinpoint whether this is network path, concurrency, or service-side tail latency.
