�� - Encoding Issues with Streaming Responses from OpenAI API in Edge Runtime

API Bugs
1

Hello everyone,

I’m encountering an issue with character encoding when streaming responses from the OpenAI API in an Edge runtime environment. Despite setting the correct content type to UTF-8, special characters (such as Hebrew) are getting corrupted and displayed as “��” in the response.

Here is a summary of my setup:

  • I’m using the openai.chat.completions.create method to stream responses.
  • The responses are processed and streamed back to the client using a ReadableStream.
  • The Content-Type header is set to text/plain; charset=UTF-8.

However, despite these settings, special characters are still not being handled correctly. I have tried using TextEncoder and TextDecoder, but the issue persists.

Has anyone experienced similar issues or have any insights on how to handle UTF-8 encoding properly in this context? Any help or guidance would be greatly appreciated!

Thank you in advance!

3 Likes