pylupdate5 forgets translations of lines containing Unicode

[Phil Thompson]

pylupdate5 assumes a latin1 encoding unless there is an explicit 
'encoding' attribute. This is based on a very old implementation of 
lupdate. Current versions of lupdate seem to assume that the encoding is 
UTF-8 (as does pylupdate6).

Although changing the behaviour of pylupdate5 at this late stage of its 
life make me feel a bit uncomfortable, the current behaviour is 
definitely wrong. I think the corresponding change in Qt was v5.6.

Anyway, it should be fixed in the next snapshot.

Thanks,
Phil

On 14/04/2021 07:37, Anton Yablokov wrote:
> I've conducted some research. It appears that for non-ASCII lines, 
> <message>
> XML tag of TS files should have encoding="UTF-8" attribute. For some
> reason, PyQt5 Linguist 5.12.8 I have omits the attribute. Without it, 
> the
> translation gets lost.
> 
> I can't really dig much deeper into the problem. Is it fixed in newer
> Linguist versions?
> 🍏
> 
> 
> On Mon, Apr 12, 2021 at 8:26 PM Anton Yablokov <stsav012 at gmail.com> 
> wrote:
> 
>> Dear community,
>> 
>> I've just bumped into a troubling behavior of pylupdate5. Namely, it
>> marks the translations of lines containing Unicode symbols besides 
>> ASCII as
>> unfinished and actually removes the translation.
>> 
>> Here is an MWE:
>> 
>> from PyQt5.QtCore import QCoreApplication
>> 
>> _translate = QCoreApplication.translate
>> _translate('microvolts', 'uV')
>> _translate('microvolts', 'μV')
>> 
>> 
>> On it, run
>> 
>>  pylupdate5 -noobsolete *.py -ts microvolts.ts
>> 
>> once, then complete (and save) the translation using linguist, then 
>> run
>> pylupdate5 again with the same parameters. The translation of
>> 
>> 'μV'
>> 
>> vanishes:
>> 
>> <         <translation>μV</translation>
>> ---
>> >         <translation type="unfinished"></translation>
>> 
>> Checked with PyQt5 5.13.2 through 5.15.4 on Linux.
>> 
>> Best regards,
>> Anton Yablokov
>> 🍏
>>