When ChatGPT is creating or editing documents and a backend file-generation service becomes rate-limited, the user should not have to manually retry the request.
Instead, ChatGPT should:
- Queue the document job rather than reject it.
- Automatically retry with exponential backoff.
- Fail over to another available file-generation worker or region when possible.
- Preserve the user’s exact document-edit instructions until the file is successfully produced.
- Use a lightweight execution path for simple edits, such as changing a few lines in a small HTML or document file.
From the user’s perspective, the experience should simply be:
“Creating file…” → “File ready.”
This would prevent users from seeing “Too Many Requests” errors during normal document creation and would make ChatGPT’s document workflow much more reliable.
In my case, the task was only to make two small changes to a simple HTML invoice, yet the file-generation service repeatedly returned a rate-limit error. The requested edits were clear and lightweight, so this seems like a good use case for queuing, automatic retry, or failover.