Streaming responses work well in my local development device (in “python manage.py runserver”). However, it does not in the web-hosted version. Let me elaborate.
The total time it takes for the response to be fully made is the same. But the size of the response chunk per stream is much bigger in the web-hosted version. This makes it seem the web-hosted version to be “slower”.
For example, it looks like the following:
In the local version: streams 1~3 words at a time
In the web version: streams 20 words at a time
→ since it takes longer to generate 20 words (than 1~3 words), streaming seems “slower”.
Any experiences/solutions that I can get help with?
The issue was related to buffering in the web server (Nginx) when it was interfacing with the uWSGI server, which was running my Django application.
Nginx Buffering:
By default, Nginx buffers responses from proxied servers (like uWSGI) before sending them to the client. This behavior is generally beneficial because it can reduce the load on the network and the number of read/write operations. However, for real-time features like Server-Sent Events (SSE), this buffering can lead to delays because Nginx waits to collect a certain amount of data before sending it out.
Solution:
Disabling Nginx’s buffering for the specific location block where the SSE was happening (location /) ensured that Nginx sent each piece of data as soon as it was received from the uWSGI server, without waiting to buffer more data. This is crucial for a real-time streaming feature to function correctly, as it minimizes latency between the server’s response generation and the client’s receipt of that response.