I can ask an AI to write it for me, saving me writing what I know but don’t actually know:
These characters are often used for bidirectional text (e.g., mixing English and Arabic) and controlling text direction within a document. I’ll also provide information on how they can be used programmatically in Python to change text direction.
Character
Description
Usage in Python
U+202A () LRE
Left-to-Right Embedding
text = '\u202A' + text (Wrap text to display as left-to-right)
U+202B () RLE
Right-to-Left Embedding
text = '\u202B' + text (Wrap text to display as right-to-left)
U+202C () PDF
Pop Directional Formatting
text = '\u202C' + text (Pop direction formatting)
U+200E () LRM
Left-to-Right Mark
text = '\u200E' + text (Set base direction to left-to-right)
U+200F () RLM
Right-to-Left Mark
text = '\u200F' + text (Set base direction to right-to-left)
U+202D () LRO
Left-to-Right Override
text = '\u202D' + text (Force left-to-right embedding)
U+202E () RLO
Right-to-Left Override
text = '\u202E' + text (Force right-to-left embedding)
These characters can be used in Python to control text direction programmatically. For example, if you receive text that you want to display as right-to-left (e.g., Arabic), you can wrap the text with the appropriate control character:
python
text = '\u202B' + arabic_text # Right-to-Left Embedding
print(text)
Conversely, if you want to switch the direction back to left-to-right (e.g., English) within the same document, you can use the Pop Directional Formatting character:
python
text = '\u202C' + english_text # Pop Directional Formatting
print(text)
These characters allow you to control the direction of text within a document, especially when dealing with mixed-direction text.
Please note that handling bidirectional text and text direction in Unicode can be complex, and the usage of these characters may vary depending on your specific requirements and the text processing library you are using in Python.
TL:DR. Never figured out how to encode/decode the response, but found a workaround simply using PS7.
I was facing a similar issue using a PowerShell script which essentially uses the API with Invoke-RestMethod. When my prompt generated a response that has non-standard characters, I’d get back the garbled text. For ex.
Me: How do you say “what” in Japanese?
AI: In Japanese, “what” is translated as “ä½” (nani) or “ãªã” (nan).
In the end, I didn’t figure out a way to properly display the text but through research suspected it was an issue with PS 5. When I tried the script in PS 7, it worked!
AI: In Japanese, “what” is typically translated as “nani” (何).
Additionally, I learned that if I wanted to SEND any non-English characters in my prompt to ChatGPT, I had to add two things, the uft-8 to content-type… "Content-Type" = "application/json; charset=utf-8"
and encode my input hashtable. $body = [System.Text.Encoding]::UTf8.GetBytes($body)
Btw, the script I’m using is the conversation one here:
github_com/yzwijsen/chatgpt-powershell
