Ui file encoding issue: PyQt5.uic.compileUiDir() uses local default text encoding for all UI files, instead of utf-8 (PyQt5 5.15.7)

[Thomas Hess]

Hello list,

I think I found a file-encoding related bug in PyQt5.uic.compileUiDir():

It internally open()s the input UI file and output Python file without 
specifying the encoding, thus using the system-default encoding for text files.

For my Windows VMs, the default encoding is cp1252. Compiling my UI files 
causes a UnicodeDecodeError on utf-8 encoded UI files that contain characters 
invalid in said default encoding, which crashes the ui compilation step.

Additionally, the generated file header explicitly mentions that the output is 
using utf-8, which is currently simply not true, because compileUiDir() 
doesn’t always write using utf-8.

I propose this patch (released as Public Domain) to solve both points (which 
is also attached):


--- a/PyQt5/uic/__init__.py
+++ b/PyQt5/uic/__init__.py
@@ -111,13 +111,11 @@ def compileUiDir(dir, recurse=False, map=None, 
**compileUi_args):
             ui_path = os.path.join(ui_dir, ui_file)
             py_path = os.path.join(py_dir, py_file)
 
-            ui_file = open(ui_path, 'r')
-            py_file = open(py_path, 'w')
+            py_file = open(py_path, 'w',  encoding='utf-8')
 
             try:
-                compileUi(ui_file, py_file, **compileUi_args)
+                compileUi(ui_path, py_file, **compileUi_args)
             finally:
-                ui_file.close()
                 py_file.close()
 
For the output side, the patched code explicitly specifies utf-8 as the output 
encoding, according to the header defined starting with line 46.

The input encoding issue is solved by not opening the UI file here at all, 
instead passing the file path to compileUi(), which can handle file paths 
according to the docstring. The compileUi-internal file handling logic then 
handles the utf-8 input files properly.

The attached UI file contains a minimal crashing example. Place mce.ui into the 
the current directory.
Compiling via compileUi is successful: 
>>> with open("mce.py", "w", encoding="utf-8") as f: compileUi("mce.ui", f)

Compiling via compileUiDir fails:
>>> compileUiDir(".")
[snip]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 161: 
character maps to <undefined>

Greetings,

Thomas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: compileUiDir.patch
Type: text/x-patch
Size: 650 bytes
Desc: not available
URL: <https://www.riverbankcomputing.com/pipermail/pyqt/attachments/20221019/3c99f484/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mce.ui
Type: application/x-designer
Size: 233 bytes
Desc: not available
URL: <https://www.riverbankcomputing.com/pipermail/pyqt/attachments/20221019/3c99f484/attachment-0001.bin>