[PyQt] QApplication constructor mangles non-ascii characters in unicode strings

[Phil Thompson]

On 12/01/2015 8:59 pm, Pavel Roskin wrote:
> Hello!
> 
> I'm porting a PyQt based application to Python 3. If I initialize
> QApplication with sys.argv, QApplication.arguments() returns those
> arguments with non-ascii characters stripped out in PyQt4 or replaced
> with "replacement character" in PyQt5. The end result is that files
> with non-ascii (but valid UTF-8) names cannot be opened. A workaround
> that I don't quite like is to encode the arguments and pass them as
> bytes.
> 
> I made this simple test case to show the problem. The test case runs
> in both Python 2 and Python 3. Comment out the other import to use
> PyQt4 instead of PyQt5.
> 
> #! /usr/bin/python
> #from PyQt4.QtGui import QApplication
> from PyQt5.QtWidgets import QApplication
> args = [u"test", u"tsch\u00fcss", u"tsch\u00fcss".encode("utf-8")]
> print("args:", args)
> qapp = QApplication(args)
> qargs = qapp.arguments()
> for qarg in qargs:
>     print("qarg:", qarg)
> 
> Output under Python 2 and PyQt5
> 
> ('args:', [u'test', u'tsch\xfcss', 'tsch\xc3\xbcss'])
> ('qarg:', u'test')
> ('qarg:', u'tsch\ufffdss')
> ('qarg:', u'tsch\xfcss')
> 
> \ufffd is a the unicode replacement character. It cannot be traced
> back to u umlaut. \xfc is u umlaut. So the unicode argument got
> mangled, whereas the encoded argument was decoded back to correct
> unicode.
> 
> I just want to pass sys.argv to the QApplication constructor without
> any extra manipulations, with any Python version.
> 
> I'm using Ubuntu 14.10 with the following packages:
> 
> python-qt4 4.11.2+dfsg-1
> python-pyqt5 5.3.2+dfsg-1ubuntu1
> python3-pyqt4 4.11.2+dfsg-1
> python3-pyqt5 5.3.2+dfsg-1ubuntu1
> python 2.7.8-1
> python3 3.4.2-1

Should be fixed in tonight's snapshots.

Thanks,
Phil