From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A5DBC433F5 for ; Thu, 16 Sep 2021 15:58:18 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6E0CA6120E for ; Thu, 16 Sep 2021 15:58:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 6E0CA6120E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=nongnu.org Received: from localhost ([::1]:50076 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mQtmC-0000uL-Fu for qemu-devel@archiver.kernel.org; Thu, 16 Sep 2021 11:58:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43942) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQtVM-0003kS-UE for qemu-devel@nongnu.org; Thu, 16 Sep 2021 11:40:52 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:32208) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mQtVJ-0002Vc-Iv for qemu-devel@nongnu.org; Thu, 16 Sep 2021 11:40:52 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1631806849; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=M66nfg5/9svfKvLf/f7AIeZQFYSYlVUiKuZsZqx8DRw=; b=TyPXPhrTmFetDY76B4rrdQyKyYZKgNU6jNXoW3ux2NnAqGh9pFWtputKXFmgeEyu+aREKe eUsKnDlOMPvgl3FEzeKGduFS2y0BryW6/LWZHU3vwYgyu3JLekVe+XrrPFD4dSo4nCXoqe 3+6Y8g9J1F9FW2G704ilrqFuYsg4VzM= Received: from mail-oo1-f70.google.com (mail-oo1-f70.google.com [209.85.161.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-349-d6XTWjIpO_6cbqipJWUcsA-1; Thu, 16 Sep 2021 11:40:45 -0400 X-MC-Unique: d6XTWjIpO_6cbqipJWUcsA-1 Received: by mail-oo1-f70.google.com with SMTP id m4-20020a4ae3c4000000b00291a653baceso30866706oov.21 for ; Thu, 16 Sep 2021 08:40:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=M66nfg5/9svfKvLf/f7AIeZQFYSYlVUiKuZsZqx8DRw=; b=J2C9PsVul3zTjPUWDtee9bJNI9EQdGDZL4tOvhgNb6E5VDrjyYbIpTGVYWgaE6+iVr ekIsOl3U1juTgx3jZ3Te8zzSzjr7MvEgcvQdd5VctEA+OKV+wBitrCs6hc0A/l6hv+kk BvrZCnaK266Lnbd0GL8zdlrhrZxl/Hojfk3OPrJrsSHjZQuxy7aQvYDkVuBW4IAecKAQ 7y+NhmRdTZnxBgjzbPUeBSj3z6NW0CWeA0EMonKln7cy4kp9Y8m+I9cyxOjXW2mPL+Ag mE6ymXIdcRSZWAIi98MY3fd/kFV8EXcjPOuZJHKKqU3dPxvaFC6qehA2ldXm6WRZlxwF gILw== X-Gm-Message-State: AOAM533OCCq4vL9vVpyswkkdDMEU/znx4xQlkmpwxqinLiXz8DzDa7lj 7yQGi/IwS1olReJYFMemHw9w/wsT61usqJtHm70gLgvsX1DzDqNX4yXtaHCSXhqHkoOt1JJ9kAD 946p7KeP7dMPN/A7/xa0tUeeBdpl0k7o= X-Received: by 2002:aca:3887:: with SMTP id f129mr2216213oia.112.1631806844920; Thu, 16 Sep 2021 08:40:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyepQbycICIe52Bf3m1Aq9838GHWIB5Q4rF8vWsRD/M0eVDXPtKH6zeFXsjMj24/f8/F3N3JUZfk8ghICZS7DQ= X-Received: by 2002:aca:3887:: with SMTP id f129mr2216192oia.112.1631806844681; Thu, 16 Sep 2021 08:40:44 -0700 (PDT) MIME-Version: 1.0 References: <20210915154031.321592-1-jsnow@redhat.com> <20210915154031.321592-2-jsnow@redhat.com> In-Reply-To: From: John Snow Date: Thu, 16 Sep 2021 11:40:34 -0400 Message-ID: Subject: Re: [PATCH v3 1/1] python: Update for pylint 2.10 To: =?UTF-8?Q?Daniel_P=2E_Berrang=C3=A9?= Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jsnow@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: multipart/alternative; boundary="00000000000051d3e505cc1ea36b" Received-SPF: pass client-ip=216.205.24.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -31 X-Spam_score: -3.2 X-Spam_bar: --- X-Spam_report: (-3.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.392, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , Eduardo Habkost , qemu-devel , G S Niteesh Babu , Cleber Rosa , Eric Blake Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" --00000000000051d3e505cc1ea36b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Sep 16, 2021 at 11:39 AM Daniel P. Berrang=C3=A9 wrote: > On Thu, Sep 16, 2021 at 09:42:30AM -0400, John Snow wrote: > > On Thu, Sep 16, 2021 at 8:59 AM Daniel P. Berrang=C3=A9 > > wrote: > > > > > On Wed, Sep 15, 2021 at 11:40:31AM -0400, John Snow wrote: > > > > A few new annoyances. Of note is the new warning for an unspecified > > > > encoding when opening a text file, which actually does indicate a > > > > potentially real problem; see > > > > https://www.python.org/dev/peps/pep-0597/#motivation > > > > > > > > Use LC_CTYPE to determine an encoding to use for interpreting QEMU'= s > > > > terminal output. Note that Python states: "language code and encodi= ng > > > > may be None if their values cannot be determined" -- use a platform > > > > default as a backup. > > > > > > > > Signed-off-by: John Snow > > > > --- > > > > python/qemu/machine/machine.py | 9 ++++++++- > > > > python/setup.cfg | 1 + > > > > 2 files changed, 9 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/python/qemu/machine/machine.py > > > b/python/qemu/machine/machine.py > > > > index a7081b1845..51b6e79a13 100644 > > > > --- a/python/qemu/machine/machine.py > > > > +++ b/python/qemu/machine/machine.py > > > > @@ -19,6 +19,7 @@ > > > > > > > > import errno > > > > from itertools import chain > > > > +import locale > > > > import logging > > > > import os > > > > import shutil > > > > @@ -290,8 +291,14 @@ def get_pid(self) -> Optional[int]: > > > > return self._subp.pid > > > > > > > > def _load_io_log(self) -> None: > > > > + # Assume that the output encoding of QEMU's terminal outpu= t > > > > + # is defined by our locale. If indeterminate, use a platfo= rm > > > default. > > > > + _, encoding =3D locale.getlocale() > > > > + if encoding is None: > > > > + encoding =3D > locale.getpreferredencoding(do_setlocale=3DFalse) > > > > > > Do we really need this getpreferredencoding ? IIUC, this is a sign > > > that the application is buggy by not calling > > > > > > locale.setlocale(locale.LC_ALL, '') > > > > > > during its main() method, which I think we can just delegate to the > > > code in question to fix. Missing setlocale will affect everything > > > they run, so doing workarounds in only 1 place is not worth it IMHO > > > > > > > > I genuinely don't know! (And, I try to keep the Python code free from > > assuming Linux as much as I can help it.) > > > > Python's getlocale documentation states: "language code and encoding ma= y > be > > None if their values cannot be determined." > > https://docs.python.org/3/library/locale.html#locale.getlocale > > > > But it is quiet as to the circumstances under which this may happen. > > Browsing the cpython source code, (3.9ish): > > > > ``` > > def getlocale(category=3DLC_CTYPE): > > localename =3D _setlocale(category) > > if category =3D=3D LC_ALL and ';' in localename: > > raise TypeError('category LC_ALL is not supported') > > return _parse_localename(localename) > > ``` > > _setlocale is ultimately a call to (I think) _localemodule.c's > > PyLocale_setlocale(PyObject *self, PyObject *args) C function. > > It calls `result =3D setlocale(category, locale)` where the category is > going > > to be LC_CTYPE, so this should be equivalent to locale(3) (LC_CTYPE, > NULL). > > > > locale(3) says that "The return value is NULL if the request cannot be > > honored." > > > > Python parses that string according to _parse_localename, which in turn > > calls normalize(localename). > > Normalization looks quite involved, but has a fallback of returning the > > string verbatim. If the normalized locale string is "C", we return the > > tuple (None, None)! > > > > So I figured there was a non-zero chance that we'd see a value of `None= ` > > here. > > > > Source code is in cpython/Lib/locale.py and > cpython/Modules/_localemodule.c > > if you want to nose around yourself. > > > > I also have no idea how this will all shake out on Windows, so I decide= d > to > > add the fallback here just in case. (Does the Python package work on > > Windows? I don't know, but I avoid assuming it won't EVER run there... > > Certainly, I have an interest in having the QMP packages I am building > work > > on all platforms.) > > > > Thoughts? > > Well this machine.py is using UNIX domain sockets and FD passing, > so Windows is out of the question. > > I'd be inclined to just keep it simple unless someone reports a > real bug with it. > > What about the case where getlocale finds "C" and can't/won't return an encoding? Is that not a real circumstance? --js --00000000000051d3e505cc1ea36b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Sep 16, 2021 at 11:39 AM Dani= el P. Berrang=C3=A9 <berrange@red= hat.com> wrote:
On Thu, Sep 16, 2021 at 09:42:30AM -0400, John Snow wrote:
> On Thu, Sep 16, 2021 at 8:59 AM Daniel P. Berrang=C3=A9 <berrange@redhat.com><= br> > wrote:
>
> > On Wed, Sep 15, 2021 at 11:40:31AM -0400, John Snow wrote:
> > > A few new annoyances. Of note is the new warning for an unsp= ecified
> > > encoding when opening a text file, which actually does indic= ate a
> > > potentially real problem; see
> > > https://www.python.org/dev/peps/p= ep-0597/#motivation
> > >
> > > Use LC_CTYPE to determine an encoding to use for interpretin= g QEMU's
> > > terminal output. Note that Python states: "language cod= e and encoding
> > > may be None if their values cannot be determined" -- us= e a platform
> > > default as a backup.
> > >
> > > Signed-off-by: John Snow <jsnow@redhat.com>
> > > ---
> > >=C2=A0 python/qemu/machine/machine.py | 9 ++++++++-
> > >=C2=A0 python/setup.cfg=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0| 1 +
> > >=C2=A0 2 files changed, 9 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/python/qemu/machine/machine.py
> > b/python/qemu/machine/machine.py
> > > index a7081b1845..51b6e79a13 100644
> > > --- a/python/qemu/machine/machine.py
> > > +++ b/python/qemu/machine/machine.py
> > > @@ -19,6 +19,7 @@
> > >
> > >=C2=A0 import errno
> > >=C2=A0 from itertools import chain
> > > +import locale
> > >=C2=A0 import logging
> > >=C2=A0 import os
> > >=C2=A0 import shutil
> > > @@ -290,8 +291,14 @@ def get_pid(self) -> Optional[int]:<= br> > > >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return self._subp.pid
> > >
> > >=C2=A0 =C2=A0 =C2=A0 def _load_io_log(self) -> None:
> > > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 # Assume that the output encodi= ng of QEMU's terminal output
> > > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 # is defined by our locale. If = indeterminate, use a platform
> > default.
> > > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 _, encoding =3D locale.getlocal= e()
> > > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 if encoding is None:
> > > +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 encoding =3D loca= le.getpreferredencoding(do_setlocale=3DFalse)
> >
> > Do we really need this getpreferredencoding ?=C2=A0 IIUC, this is= a sign
> > that the application is buggy by not calling
> >
> >=C2=A0 =C2=A0locale.setlocale(locale.LC_ALL, '')
> >
> > during its main() method, which I think we can just delegate to t= he
> > code in question to fix. Missing setlocale will affect everything=
> > they run, so doing workarounds in only 1 place is not worth it IM= HO
> >
> >
> I genuinely don't know! (And, I try to keep the Python code free f= rom
> assuming Linux as much as I can help it.)
>
> Python's getlocale documentation states: "language code and e= ncoding may be
> None if their values cannot be determined."
> https://docs.python.org/3/library= /locale.html#locale.getlocale
>
> But it is quiet as to the circumstances under which this may happen. > Browsing the cpython source code, (3.9ish):
>
> ```
> def getlocale(category=3DLC_CTYPE):
>=C2=A0 =C2=A0 =C2=A0localename =3D _setlocale(category)
>=C2=A0 =C2=A0 =C2=A0if category =3D=3D LC_ALL and ';' in locale= name:
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0raise TypeError('category LC_ALL = is not supported')
>=C2=A0 =C2=A0 =C2=A0return _parse_localename(localename)
> ```
> _setlocale is ultimately a call to (I think) _localemodule.c's
> PyLocale_setlocale(PyObject *self, PyObject *args) C function.
> It calls `result =3D setlocale(category, locale)` where the category i= s going
> to be LC_CTYPE, so this should be equivalent to locale(3) (LC_CTYPE, N= ULL).
>
> locale(3) says that "The return value is NULL if the request cann= ot be
> honored."
>
> Python parses that string according to _parse_localename, which in tur= n
> calls normalize(localename).
> Normalization looks quite involved, but has a fallback of returning th= e
> string verbatim. If the normalized locale string is "C", we = return the
> tuple (None, None)!
>
> So I figured there was a non-zero chance that we'd see a value of = `None`
> here.
>
> Source code is in cpython/Lib/locale.py and cpython/Modules/_localemod= ule.c
> if you want to nose around yourself.
>
> I also have no idea how this will all shake out on Windows, so I decid= ed to
> add the fallback here just in case. (Does the Python package work on > Windows? I don't know, but I avoid assuming it won't EVER run = there...
> Certainly, I have an interest in having the QMP packages I am building= work
> on all platforms.)
>
> Thoughts?

Well this machine.py is using UNIX domain sockets and FD passing,
so Windows is out of the question.

I'd be inclined to just keep it simple unless someone reports a
real bug with it.


What about the case where getlocale fi= nds "C" and can't/won't return an encoding? Is that not a= real circumstance?

--js
--00000000000051d3e505cc1ea36b--