[PyQt] UnicodeDecodeError with output from Windows OS command

[Florian Bruhin]

On Thu, Nov 30, 2017 at 11:39:22AM +0000, J Barchan wrote:
> ​Oh, BTW, I meant to sy: you might want to look at
> https://stackoverflow.com/questions/12468179/unicodedecodeerror-utf8-codec-cant-decode-byte-0x9c,
> which shows I am not alone in this issue from Python/PyQt.

Because encodings are just a fact of life. You can't display binary data as
text when you don't know the encoding, you can only *guess* (often wrongly).

> And if I understand it right, nobody came up with a robust solution from
> Python.  Which is crazy, because as I keep saying, I have never had any
> problem outside Python/PyQy/Qt treating it all as just bytes and not doing
> any kind of decoding.  I don't see why I can't do that, and I don't
> understand why I have to get involved in ​decoding now when I've never had
> to.  If PyQt didn't hide QByteArray & QString from me and insist I have to
> do it via native Python bytes & str types I don't see I would have any
> problems?

If you want to display bytes as text, you need to decode them.

If you didn't have to deal with that in other contexts, it's probably because
something did guess for you, or just tried something and it *happened* to work
as long as you're only dealing with ASCII. It'll blow up as soon as that's not
the case anymore though, or display garbage.

And no, even for Qt's QByteArray and QString you have different QString::from*
methods depending on the encoding. The difference is that those fail silently
by default:

> However, invalid sequences are possible with UTF-8 and, if any such are
> found, they will be replaced with one or more "replacement characters", or
> suppressed. 

Like I said earlier - Python 3 decided that failing silently (and just
corrupting data without the programmer knowing) is a very bad thing to do, and
I fully agree.

Florian

-- 
https://www.qutebrowser.org  | me at the-compiler.org (Mail/XMPP)
   GPG: 916E B0C8 FD55 A072  | https://the-compiler.org/pubkey.asc
         I love long mails!  | https://email.is-not-s.ms/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://www.riverbankcomputing.com/pipermail/pyqt/attachments/20171130/4b0038a6/attachment.sig>