Python code automatically converts characters to ASCIi

Prompting
1

A smartphone screenshot shows a "Select Text" screen displaying a Python code snippet for reading and cleaning a file using regular expressions. (Captioned by AI)
A smartphone screenshot shows a "Select Text" screen displaying a Python code snippet for reading and cleaning a file using regular expressions. (Captioned by AI)480×960 15.2 KB

cleaned_content = re.sub(r’\x1b

 notice the ASCII characters after =re sub(r\x1b

??? The code is supposed to be standard UTF-8
BUT
somewhere between Chat-GPTs neural network &“mind && fingertips” the code / response is being converted to ASCII characters which pydroid3, Termux & other tools don’t interpret
sooo when Chat-GPT sends me the corrected version it still is in ASCII

I’ve tried running it on play store app, in chrome, Firefox etc.

this is problematic probably for a lot of people. :black_small_square:
Even when I output clean UTF-8-compliant Python, some characters — especially escaped sequences like \x1b, \u, \r, \n, or even backslashes — get silently corrupted in transit:

  • Android input methods (IME) may normalize or escape characters
  • Chrome/Firebase/Clipboard APIs sometimes auto-convert or truncate escape sequences
  • TextView components inside apps like Pydroid3 re-interpret \x as literal control characters instead of raw text
  • Some fonts in Android may misrender invisible control glyphs

This is why what looks like:

cleaned_content = re.sub(r'\x1b
2

Why u do that?

import re

input_path = '/storage/emulated/0/Android/data/ru.iiec.pydroid3/files/.bashrc.py'
output_path = '/storage/emulated/0/Android/data/ru.iiec.pydroid3/files/.bashrc_cleaned.py'

with open(input_path, 'r', encoding='utf-8', errors='ignore') as f:
    content = f.read()

cleaned_content = re.sub(r'\x1b

That is an escape character. chr(27)

Being in the first 127 code points of UTF-8, ASCII and UTF-8 are the same single bytes.

I don’t know where the incomplete line you show came from. However, I can answer the “why” it could be there or could be predicted.

The ESC (\x1b) character is used to introduce ANSI escape sequences for text formatting and terminal control, which are also both ASCII and UTF-8.

print("\x1b[1mThis is bold text.\x1b[0m")

I have a streaming markdown to color reformatter that even employs them.

The AI shouldn’t generally be making them. It writes in readable text, markdown. You could strip all bytes 0-31 (apart from tabs or linefeeds), but then were it actually ANSI control that continued, or even printer control, you’d be left with the remainder.

Can you explain more clearly what you think the problem is and where it is coming from, and the resulting symptom?