QEMU-Devel Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH] qapi: Fix code generation with Python 3.5
@ 2020-01-16 20:25 Markus Armbruster
  2020-01-16 21:17 ` John Snow
                   ` (3 more replies)
  0 siblings, 4 replies; 14+ messages in thread
From: Markus Armbruster @ 2020-01-16 20:25 UTC (permalink / raw)
  To: qemu-devel; +Cc: thuth, mdroth

Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
modules" switched QAPISchema.visit() from

    for entity in self._entity_list:

effectively to

    for mod in self._module_dict.values():
        for entity in mod._entity_list:

Visits in the same order as long as .values() is in insertion order.
That's the case only for Python 3.6 and later.  Before, it's in some
arbitrary order, which results in broken generated code.

Fix by making self._module_dict an OrderedDict rather than a dict.

Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
Signed-off-by: Markus Armbruster <armbru@redhat.com>
---
 scripts/qapi/schema.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
index 0bfc5256fb..5100110fa2 100644
--- a/scripts/qapi/schema.py
+++ b/scripts/qapi/schema.py
@@ -795,7 +795,7 @@ class QAPISchema(object):
         self.docs = parser.docs
         self._entity_list = []
         self._entity_dict = {}
-        self._module_dict = {}
+        self._module_dict = OrderedDict()
         self._schema_dir = os.path.dirname(fname)
         self._make_module(None) # built-ins
         self._make_module(fname)
-- 
2.21.1



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] qapi: Fix code generation with Python 3.5
  2020-01-16 20:25 [PATCH] qapi: Fix code generation with Python 3.5 Markus Armbruster
@ 2020-01-16 21:17 ` John Snow
  2020-01-17  7:07   ` Markus Armbruster
  2020-01-17  9:34 ` [PATCH] qapi: Fix code generation with Python 3.5 Thomas Huth
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 14+ messages in thread
From: John Snow @ 2020-01-16 21:17 UTC (permalink / raw)
  To: Markus Armbruster, qemu-devel; +Cc: thuth, mdroth



On 1/16/20 3:25 PM, Markus Armbruster wrote:
> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
> modules" switched QAPISchema.visit() from
> 
>     for entity in self._entity_list:
> 
> effectively to
> 
>     for mod in self._module_dict.values():
>         for entity in mod._entity_list:
> 
> Visits in the same order as long as .values() is in insertion order.
> That's the case only for Python 3.6 and later.  Before, it's in some
> arbitrary order, which results in broken generated code.
> 
> Fix by making self._module_dict an OrderedDict rather than a dict.
> 
> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  scripts/qapi/schema.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
> index 0bfc5256fb..5100110fa2 100644
> --- a/scripts/qapi/schema.py
> +++ b/scripts/qapi/schema.py
> @@ -795,7 +795,7 @@ class QAPISchema(object):
>          self.docs = parser.docs
>          self._entity_list = []
>          self._entity_dict = {}
> -        self._module_dict = {}
> +        self._module_dict = OrderedDict()
>          self._schema_dir = os.path.dirname(fname)
>          self._make_module(None) # built-ins
>          self._make_module(fname)
> 

This problem has bitten me *many* times. I'm wondering if there's a
prescription that isn't just "Wait until we can stipulate 3.6+".



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] qapi: Fix code generation with Python 3.5
  2020-01-16 21:17 ` John Snow
@ 2020-01-17  7:07   ` Markus Armbruster
  2020-01-17 19:41     ` John Snow
  0 siblings, 1 reply; 14+ messages in thread
From: Markus Armbruster @ 2020-01-17  7:07 UTC (permalink / raw)
  To: John Snow
  Cc: Peter Maydell, thuth, Eduardo Habkost, Cleber Rosa, qemu-devel, mdroth

John Snow <jsnow@redhat.com> writes:

> On 1/16/20 3:25 PM, Markus Armbruster wrote:
>> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
>> modules" switched QAPISchema.visit() from
>> 
>>     for entity in self._entity_list:
>> 
>> effectively to
>> 
>>     for mod in self._module_dict.values():
>>         for entity in mod._entity_list:
>> 
>> Visits in the same order as long as .values() is in insertion order.
>> That's the case only for Python 3.6 and later.  Before, it's in some
>> arbitrary order, which results in broken generated code.
>> 
>> Fix by making self._module_dict an OrderedDict rather than a dict.
>> 
>> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>  scripts/qapi/schema.py | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>> 
>> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
>> index 0bfc5256fb..5100110fa2 100644
>> --- a/scripts/qapi/schema.py
>> +++ b/scripts/qapi/schema.py
>> @@ -795,7 +795,7 @@ class QAPISchema(object):
>>          self.docs = parser.docs
>>          self._entity_list = []
>>          self._entity_dict = {}
>> -        self._module_dict = {}
>> +        self._module_dict = OrderedDict()
>>          self._schema_dir = os.path.dirname(fname)
>>          self._make_module(None) # built-ins
>>          self._make_module(fname)
>> 
>
> This problem has bitten me *many* times. I'm wondering if there's a
> prescription that isn't just "Wait until we can stipulate 3.6+".

No clue.

3.5 EOL is scheduled for 2020-09-13.
https://devguide.python.org/#status-of-python-branches

We support 3.5 because we support Debian 9.

We'd normally drop support for Debian 9 two years after Debian 10,
i.e. July 2021.  Assuming Debian supports it that far.  Whether they can
truly support Python 3.5 after uptstream EOL seems doubtful.



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] qapi: Fix code generation with Python 3.5
  2020-01-16 20:25 [PATCH] qapi: Fix code generation with Python 3.5 Markus Armbruster
  2020-01-16 21:17 ` John Snow
@ 2020-01-17  9:34 ` Thomas Huth
  2020-01-17 10:49   ` Markus Armbruster
  2020-01-20 10:52 ` Alex Bennée
  2020-01-20 12:53 ` Peter Maydell
  3 siblings, 1 reply; 14+ messages in thread
From: Thomas Huth @ 2020-01-17  9:34 UTC (permalink / raw)
  To: Markus Armbruster, qemu-devel, Peter Maydell; +Cc: Alex Bennée, mdroth

On 16/01/2020 21.25, Markus Armbruster wrote:
> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
> modules" switched QAPISchema.visit() from
> 
>     for entity in self._entity_list:
> 
> effectively to
> 
>     for mod in self._module_dict.values():
>         for entity in mod._entity_list:
> 
> Visits in the same order as long as .values() is in insertion order.
> That's the case only for Python 3.6 and later.  Before, it's in some
> arbitrary order, which results in broken generated code.
> 
> Fix by making self._module_dict an OrderedDict rather than a dict.
> 
> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> ---
>  scripts/qapi/schema.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
> index 0bfc5256fb..5100110fa2 100644
> --- a/scripts/qapi/schema.py
> +++ b/scripts/qapi/schema.py
> @@ -795,7 +795,7 @@ class QAPISchema(object):
>          self.docs = parser.docs
>          self._entity_list = []
>          self._entity_dict = {}
> -        self._module_dict = {}
> +        self._module_dict = OrderedDict()
>          self._schema_dir = os.path.dirname(fname)
>          self._make_module(None) # built-ins
>          self._make_module(fname)
> 

Thanks, this fixes the problems on Travis for me!

Tested-by: Thomas Huth <thuth@redhat.com>

Peter, could you maybe apply this directly to the master branch as a
build fix?



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] qapi: Fix code generation with Python 3.5
  2020-01-17  9:34 ` [PATCH] qapi: Fix code generation with Python 3.5 Thomas Huth
@ 2020-01-17 10:49   ` Markus Armbruster
  2020-01-18  8:33     ` Philippe Mathieu-Daudé
  0 siblings, 1 reply; 14+ messages in thread
From: Markus Armbruster @ 2020-01-17 10:49 UTC (permalink / raw)
  To: Thomas Huth; +Cc: Peter Maydell, Alex Bennée, qemu-devel, mdroth

Thomas Huth <thuth@redhat.com> writes:

> On 16/01/2020 21.25, Markus Armbruster wrote:
>> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
>> modules" switched QAPISchema.visit() from
>> 
>>     for entity in self._entity_list:
>> 
>> effectively to
>> 
>>     for mod in self._module_dict.values():
>>         for entity in mod._entity_list:
>> 
>> Visits in the same order as long as .values() is in insertion order.
>> That's the case only for Python 3.6 and later.  Before, it's in some
>> arbitrary order, which results in broken generated code.
>> 
>> Fix by making self._module_dict an OrderedDict rather than a dict.
>> 
>> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>> ---
>>  scripts/qapi/schema.py | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>> 
>> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
>> index 0bfc5256fb..5100110fa2 100644
>> --- a/scripts/qapi/schema.py
>> +++ b/scripts/qapi/schema.py
>> @@ -795,7 +795,7 @@ class QAPISchema(object):
>>          self.docs = parser.docs
>>          self._entity_list = []
>>          self._entity_dict = {}
>> -        self._module_dict = {}
>> +        self._module_dict = OrderedDict()
>>          self._schema_dir = os.path.dirname(fname)
>>          self._make_module(None) # built-ins
>>          self._make_module(fname)
>> 
>
> Thanks, this fixes the problems on Travis for me!
>
> Tested-by: Thomas Huth <thuth@redhat.com>
>
> Peter, could you maybe apply this directly to the master branch as a
> build fix?

The commit message isn't quite right: s/Visits in the same order/Visits
modules in the same order/.  Peter, want me to respin for that?



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] qapi: Fix code generation with Python 3.5
  2020-01-17  7:07   ` Markus Armbruster
@ 2020-01-17 19:41     ` John Snow
  2020-01-18  6:54       ` Markus Armbruster
  2020-01-18  8:29       ` Philippe Mathieu-Daudé
  0 siblings, 2 replies; 14+ messages in thread
From: John Snow @ 2020-01-17 19:41 UTC (permalink / raw)
  To: Markus Armbruster
  Cc: Peter Maydell, thuth, Eduardo Habkost, Cleber Rosa, qemu-devel, mdroth



On 1/17/20 2:07 AM, Markus Armbruster wrote:
> John Snow <jsnow@redhat.com> writes:
> 
>> On 1/16/20 3:25 PM, Markus Armbruster wrote:
>>> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
>>> modules" switched QAPISchema.visit() from
>>>
>>>     for entity in self._entity_list:
>>>
>>> effectively to
>>>
>>>     for mod in self._module_dict.values():
>>>         for entity in mod._entity_list:
>>>
>>> Visits in the same order as long as .values() is in insertion order.
>>> That's the case only for Python 3.6 and later.  Before, it's in some
>>> arbitrary order, which results in broken generated code.
>>>
>>> Fix by making self._module_dict an OrderedDict rather than a dict.
>>>
>>> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
>>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>>> ---
>>>  scripts/qapi/schema.py | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
>>> index 0bfc5256fb..5100110fa2 100644
>>> --- a/scripts/qapi/schema.py
>>> +++ b/scripts/qapi/schema.py
>>> @@ -795,7 +795,7 @@ class QAPISchema(object):
>>>          self.docs = parser.docs
>>>          self._entity_list = []
>>>          self._entity_dict = {}
>>> -        self._module_dict = {}
>>> +        self._module_dict = OrderedDict()
>>>          self._schema_dir = os.path.dirname(fname)
>>>          self._make_module(None) # built-ins
>>>          self._make_module(fname)
>>>
>>
>> This problem has bitten me *many* times. I'm wondering if there's a
>> prescription that isn't just "Wait until we can stipulate 3.6+".
> 
> No clue.
> 
> 3.5 EOL is scheduled for 2020-09-13.
> https://devguide.python.org/#status-of-python-branches
> 
> We support 3.5 because we support Debian 9.
> 
> We'd normally drop support for Debian 9 two years after Debian 10,
> i.e. July 2021.  Assuming Debian supports it that far.  Whether they can
> truly support Python 3.5 after uptstream EOL seems doubtful.
> 

We should decide whether we consider Debian LTS to be adequately
supported, yes-or-no.

We should use a rule of "two years after successor, or End-of-Support,
whichever comes first."

For Debian, is end of support three years after it comes out, or is it
when the LTS is EOL?

In this specific case, do we trust that Debian 9 LTS will continue to
patch Python3.5 all the way up until July 2021?

--js



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] qapi: Fix code generation with Python 3.5
  2020-01-17 19:41     ` John Snow
@ 2020-01-18  6:54       ` Markus Armbruster
  2020-02-07 21:54         ` Eduardo Habkost
  2020-01-18  8:29       ` Philippe Mathieu-Daudé
  1 sibling, 1 reply; 14+ messages in thread
From: Markus Armbruster @ 2020-01-18  6:54 UTC (permalink / raw)
  To: John Snow
  Cc: Peter Maydell, thuth, Eduardo Habkost, Cleber Rosa, qemu-devel, mdroth

John Snow <jsnow@redhat.com> writes:

> On 1/17/20 2:07 AM, Markus Armbruster wrote:
>> John Snow <jsnow@redhat.com> writes:
>> 
>>> On 1/16/20 3:25 PM, Markus Armbruster wrote:
>>>> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
>>>> modules" switched QAPISchema.visit() from
>>>>
>>>>     for entity in self._entity_list:
>>>>
>>>> effectively to
>>>>
>>>>     for mod in self._module_dict.values():
>>>>         for entity in mod._entity_list:
>>>>
>>>> Visits in the same order as long as .values() is in insertion order.
>>>> That's the case only for Python 3.6 and later.  Before, it's in some
>>>> arbitrary order, which results in broken generated code.
>>>>
>>>> Fix by making self._module_dict an OrderedDict rather than a dict.
>>>>
>>>> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
>>>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>>>> ---
>>>>  scripts/qapi/schema.py | 2 +-
>>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
>>>> index 0bfc5256fb..5100110fa2 100644
>>>> --- a/scripts/qapi/schema.py
>>>> +++ b/scripts/qapi/schema.py
>>>> @@ -795,7 +795,7 @@ class QAPISchema(object):
>>>>          self.docs = parser.docs
>>>>          self._entity_list = []
>>>>          self._entity_dict = {}
>>>> -        self._module_dict = {}
>>>> +        self._module_dict = OrderedDict()
>>>>          self._schema_dir = os.path.dirname(fname)
>>>>          self._make_module(None) # built-ins
>>>>          self._make_module(fname)
>>>>
>>>
>>> This problem has bitten me *many* times. I'm wondering if there's a
>>> prescription that isn't just "Wait until we can stipulate 3.6+".
>> 
>> No clue.
>> 
>> 3.5 EOL is scheduled for 2020-09-13.
>> https://devguide.python.org/#status-of-python-branches
>> 
>> We support 3.5 because we support Debian 9.
>> 
>> We'd normally drop support for Debian 9 two years after Debian 10,
>> i.e. July 2021.  Assuming Debian supports it that far.  Whether they can
>> truly support Python 3.5 after uptstream EOL seems doubtful.
>> 
>
> We should decide whether we consider Debian LTS to be adequately
> supported, yes-or-no.
>
> We should use a rule of "two years after successor, or End-of-Support,
> whichever comes first."

Yes.

> For Debian, is end of support three years after it comes out, or is it
> when the LTS is EOL?

We need to define end-of-support for Debian: is it Debian proper or is
it Debian LTS?

<https://wiki.debian.org/DebianOldStable>:

    Q) How long will security updates be provided?

    The security team tries to support a stable distribution for about
    one year after the next stable distribution has been released,
    except when another stable distribution is released within this
    year.  It is not possible to support three distributions; supporting
    two simultaneously is already difficult enough.

<https://wiki.debian.org/LTS>:

    Debian Long Term Support (LTS) is a project to extend the lifetime
    of all Debian stable releases to (at least) 5 years.  Debian LTS is
    not handled by the Debian security team, but by a separate group of
    volunteers and companies interested in making it a success.

    Thus the Debian LTS team takes over security maintenance of the
    various releases once the Debian Security team stops its work.

Debian 10 "Buster" was released in July 2019.  Debian 9 "Stretch" will
receive security updates from Debian until mid 2020, i.e. just about
when Python 3.5 reaches EOL.  LTS will attempt to support it until June
2022.

I think we should give ourselves a bit more flexibility than the
categorical "Support for the previous major version will be dropped 2
years after the new major version is released."  At some point, the cost
of supporting old hosts exceeds the utility.  We should face this
tradeoff.

> In this specific case, do we trust that Debian 9 LTS will continue to
> patch Python3.5 all the way up until July 2021?

June 2022 even.

We use Python at compile time with trusted input.  The need for security
maintenance is relatively low there.  I'm not ready to vouch for "and we
don't use Python for anything else".



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] qapi: Fix code generation with Python 3.5
  2020-01-17 19:41     ` John Snow
  2020-01-18  6:54       ` Markus Armbruster
@ 2020-01-18  8:29       ` Philippe Mathieu-Daudé
  2020-01-20  5:46         ` [Qemu-devel] Xenial in Travis (was: qapi: Fix code generation with Python 3.5) Thomas Huth
  1 sibling, 1 reply; 14+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-01-18  8:29 UTC (permalink / raw)
  To: John Snow, Markus Armbruster, Alex Bennée
  Cc: Peter Maydell, thuth, Eduardo Habkost, Cleber Rosa, qemu-devel, mdroth

On 1/17/20 8:41 PM, John Snow wrote:
> On 1/17/20 2:07 AM, Markus Armbruster wrote:
>> John Snow <jsnow@redhat.com> writes:
>>
>>> On 1/16/20 3:25 PM, Markus Armbruster wrote:
>>>> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
>>>> modules" switched QAPISchema.visit() from
>>>>
>>>>      for entity in self._entity_list:
>>>>
>>>> effectively to
>>>>
>>>>      for mod in self._module_dict.values():
>>>>          for entity in mod._entity_list:
>>>>
>>>> Visits in the same order as long as .values() is in insertion order.
>>>> That's the case only for Python 3.6 and later.  Before, it's in some
>>>> arbitrary order, which results in broken generated code.
>>>>
>>>> Fix by making self._module_dict an OrderedDict rather than a dict.
>>>>
>>>> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
>>>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>>>> ---
>>>>   scripts/qapi/schema.py | 2 +-
>>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
>>>> index 0bfc5256fb..5100110fa2 100644
>>>> --- a/scripts/qapi/schema.py
>>>> +++ b/scripts/qapi/schema.py
>>>> @@ -795,7 +795,7 @@ class QAPISchema(object):
>>>>           self.docs = parser.docs
>>>>           self._entity_list = []
>>>>           self._entity_dict = {}
>>>> -        self._module_dict = {}
>>>> +        self._module_dict = OrderedDict()
>>>>           self._schema_dir = os.path.dirname(fname)
>>>>           self._make_module(None) # built-ins
>>>>           self._make_module(fname)
>>>>
>>>
>>> This problem has bitten me *many* times. I'm wondering if there's a
>>> prescription that isn't just "Wait until we can stipulate 3.6+".
>>
>> No clue.
>>
>> 3.5 EOL is scheduled for 2020-09-13.
>> https://devguide.python.org/#status-of-python-branches
>>
>> We support 3.5 because we support Debian 9.
>>
>> We'd normally drop support for Debian 9 two years after Debian 10,
>> i.e. July 2021.  Assuming Debian supports it that far.  Whether they can
>> truly support Python 3.5 after uptstream EOL seems doubtful.
>>
> 
> We should decide whether we consider Debian LTS to be adequately
> supported, yes-or-no.
> 
> We should use a rule of "two years after successor, or End-of-Support,
> whichever comes first."
> 
> For Debian, is end of support three years after it comes out, or is it
> when the LTS is EOL?
> 
> In this specific case, do we trust that Debian 9 LTS will continue to
> patch Python3.5 all the way up until July 2021?

This broke 29 of the 32 Travis jobs we have:

https://travis-ci.org/qemu/qemu/builds/637999366

Since we started to use Travis CI, it catched quite some bugs...

I think it is important to add in the equation we also depend of our CIs.



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] qapi: Fix code generation with Python 3.5
  2020-01-17 10:49   ` Markus Armbruster
@ 2020-01-18  8:33     ` Philippe Mathieu-Daudé
  2020-01-19 11:23       ` Philippe Mathieu-Daudé
  0 siblings, 1 reply; 14+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-01-18  8:33 UTC (permalink / raw)
  To: Markus Armbruster, Thomas Huth
  Cc: Peter Maydell, Alex Bennée, qemu-devel, mdroth

On 1/17/20 11:49 AM, Markus Armbruster wrote:
> Thomas Huth <thuth@redhat.com> writes:
> 
>> On 16/01/2020 21.25, Markus Armbruster wrote:
>>> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
>>> modules" switched QAPISchema.visit() from
>>>
>>>      for entity in self._entity_list:
>>>
>>> effectively to
>>>
>>>      for mod in self._module_dict.values():
>>>          for entity in mod._entity_list:
>>>
>>> Visits in the same order as long as .values() is in insertion order.
>>> That's the case only for Python 3.6 and later.  Before, it's in some
>>> arbitrary order, which results in broken generated code.
>>>
>>> Fix by making self._module_dict an OrderedDict rather than a dict.
>>>
>>> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
>>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>>> ---
>>>   scripts/qapi/schema.py | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
>>> index 0bfc5256fb..5100110fa2 100644
>>> --- a/scripts/qapi/schema.py
>>> +++ b/scripts/qapi/schema.py
>>> @@ -795,7 +795,7 @@ class QAPISchema(object):
>>>           self.docs = parser.docs
>>>           self._entity_list = []
>>>           self._entity_dict = {}
>>> -        self._module_dict = {}
>>> +        self._module_dict = OrderedDict()
>>>           self._schema_dir = os.path.dirname(fname)
>>>           self._make_module(None) # built-ins
>>>           self._make_module(fname)
>>>
>>
>> Thanks, this fixes the problems on Travis for me!
>>
>> Tested-by: Thomas Huth <thuth@redhat.com>

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

>> Peter, could you maybe apply this directly to the master branch as a
>> build fix?
> 
> The commit message isn't quite right: s/Visits in the same order/Visits
> modules in the same order/.  Peter, want me to respin for that?

Since it is a single patch, it shouldn't be too much work to respin :)

I agree this patch is candidate for direct fix on /master.

Thanks,

Phil.



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] qapi: Fix code generation with Python 3.5
  2020-01-18  8:33     ` Philippe Mathieu-Daudé
@ 2020-01-19 11:23       ` Philippe Mathieu-Daudé
  0 siblings, 0 replies; 14+ messages in thread
From: Philippe Mathieu-Daudé @ 2020-01-19 11:23 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé, Markus Armbruster, Thomas Huth
  Cc: Peter Maydell, Alex Bennée, qemu-devel, mdroth

On 1/18/20 9:33 AM, Philippe Mathieu-Daudé wrote:
> On 1/17/20 11:49 AM, Markus Armbruster wrote:
>> Thomas Huth <thuth@redhat.com> writes:
>>
>>> On 16/01/2020 21.25, Markus Armbruster wrote:
>>>> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
>>>> modules" switched QAPISchema.visit() from
>>>>
>>>>      for entity in self._entity_list:
>>>>
>>>> effectively to
>>>>
>>>>      for mod in self._module_dict.values():
>>>>          for entity in mod._entity_list:
>>>>
>>>> Visits in the same order as long as .values() is in insertion order.
>>>> That's the case only for Python 3.6 and later.  Before, it's in some
>>>> arbitrary order, which results in broken generated code.
>>>>
>>>> Fix by making self._module_dict an OrderedDict rather than a dict.
>>>>
>>>> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
>>>> Signed-off-by: Markus Armbruster <armbru@redhat.com>
>>>> ---
>>>>   scripts/qapi/schema.py | 2 +-
>>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
>>>> index 0bfc5256fb..5100110fa2 100644
>>>> --- a/scripts/qapi/schema.py
>>>> +++ b/scripts/qapi/schema.py
>>>> @@ -795,7 +795,7 @@ class QAPISchema(object):
>>>>           self.docs = parser.docs
>>>>           self._entity_list = []
>>>>           self._entity_dict = {}
>>>> -        self._module_dict = {}
>>>> +        self._module_dict = OrderedDict()
>>>>           self._schema_dir = os.path.dirname(fname)
>>>>           self._make_module(None) # built-ins
>>>>           self._make_module(fname)
>>>>
>>>
>>> Thanks, this fixes the problems on Travis for me!
>>>
>>> Tested-by: Thomas Huth <thuth@redhat.com>
> 
> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>

And per https://www.mail-archive.com/qemu-devel@nongnu.org/msg671745.html:

Tested-by: BALATON Zoltan <balaton@eik.bme.hu>

>>> Peter, could you maybe apply this directly to the master branch as a
>>> build fix?
>>
>> The commit message isn't quite right: s/Visits in the same order/Visits
>> modules in the same order/.  Peter, want me to respin for that?
> 
> Since it is a single patch, it shouldn't be too much work to respin :)
> 
> I agree this patch is candidate for direct fix on /master.
> 
> Thanks,
> 
> Phil.
> 
> 


^ permalink raw reply	[flat|nested] 14+ messages in thread

* [Qemu-devel] Xenial in Travis (was: qapi: Fix code generation with Python 3.5)
  2020-01-18  8:29       ` Philippe Mathieu-Daudé
@ 2020-01-20  5:46         ` Thomas Huth
  0 siblings, 0 replies; 14+ messages in thread
From: Thomas Huth @ 2020-01-20  5:46 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé,
	John Snow, Markus Armbruster, Alex Bennée
  Cc: qemu-devel, Peter Maydell, mdroth, Eduardo Habkost, Cleber Rosa

On 18/01/2020 09.29, Philippe Mathieu-Daudé wrote:
[...]
> This broke 29 of the 32 Travis jobs we have:
> 
> https://travis-ci.org/qemu/qemu/builds/637999366
> 
> Since we started to use Travis CI, it catched quite some bugs...
> 
> I think it is important to add in the equation we also depend of our CIs.

Thinking about the state of our Travis CI ... The Travis CI jobs that
broke are still running Ubuntu Xenial (v16.04). According to our support
policy, we will stop supporting Xenial two years after v18.04 LTS has
been released, i.e. in April this year. So we have to update our Travis
CI jobs to a newer version with a newer Python very soon anyway - the
question is here only whether we update to v18.04 (with the broken
libssh version that we don't support in QEMU), or rather directly to
v20.04 (which might take a while 'till it shows up as an option in
Travis)...

 Thomas



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] qapi: Fix code generation with Python 3.5
  2020-01-16 20:25 [PATCH] qapi: Fix code generation with Python 3.5 Markus Armbruster
  2020-01-16 21:17 ` John Snow
  2020-01-17  9:34 ` [PATCH] qapi: Fix code generation with Python 3.5 Thomas Huth
@ 2020-01-20 10:52 ` Alex Bennée
  2020-01-20 12:53 ` Peter Maydell
  3 siblings, 0 replies; 14+ messages in thread
From: Alex Bennée @ 2020-01-20 10:52 UTC (permalink / raw)
  To: Markus Armbruster, Peter Maydell; +Cc: thuth, mdroth, qemu-devel


Markus Armbruster <armbru@redhat.com> writes:

> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
> modules" switched QAPISchema.visit() from
>
>     for entity in self._entity_list:
>
> effectively to
>
>     for mod in self._module_dict.values():
>         for entity in mod._entity_list:
>
> Visits in the same order as long as .values() is in insertion order.
> That's the case only for Python 3.6 and later.  Before, it's in some
> arbitrary order, which results in broken generated code.
>
> Fix by making self._module_dict an OrderedDict rather than a dict.
>
> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
> Signed-off-by: Markus Armbruster <armbru@redhat.com>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>

Well that certainly clears up a bunch of red. Can we apply it directly
as a build fix?

> ---
>  scripts/qapi/schema.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/qapi/schema.py b/scripts/qapi/schema.py
> index 0bfc5256fb..5100110fa2 100644
> --- a/scripts/qapi/schema.py
> +++ b/scripts/qapi/schema.py
> @@ -795,7 +795,7 @@ class QAPISchema(object):
>          self.docs = parser.docs
>          self._entity_list = []
>          self._entity_dict = {}
> -        self._module_dict = {}
> +        self._module_dict = OrderedDict()
>          self._schema_dir = os.path.dirname(fname)
>          self._make_module(None) # built-ins
>          self._make_module(fname)


-- 
Alex Bennée


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] qapi: Fix code generation with Python 3.5
  2020-01-16 20:25 [PATCH] qapi: Fix code generation with Python 3.5 Markus Armbruster
                   ` (2 preceding siblings ...)
  2020-01-20 10:52 ` Alex Bennée
@ 2020-01-20 12:53 ` Peter Maydell
  3 siblings, 0 replies; 14+ messages in thread
From: Peter Maydell @ 2020-01-20 12:53 UTC (permalink / raw)
  To: Markus Armbruster; +Cc: Thomas Huth, QEMU Developers, Michael Roth

On Thu, 16 Jan 2020 at 20:27, Markus Armbruster <armbru@redhat.com> wrote:
>
> Recent commit 3e7fb5811b "qapi: Fix code generation for empty modules"
> modules" switched QAPISchema.visit() from
>
>     for entity in self._entity_list:
>
> effectively to
>
>     for mod in self._module_dict.values():
>         for entity in mod._entity_list:
>
> Visits in the same order as long as .values() is in insertion order.
> That's the case only for Python 3.6 and later.  Before, it's in some
> arbitrary order, which results in broken generated code.
>
> Fix by making self._module_dict an OrderedDict rather than a dict.
>
> Fixes: 3e7fb5811baab213dcc7149c3aa69442d683c26c
> Signed-off-by: Markus Armbruster <armbru@redhat.com>
> --


Applied to master as a buildfix, thanks.

-- PMM


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: [PATCH] qapi: Fix code generation with Python 3.5
  2020-01-18  6:54       ` Markus Armbruster
@ 2020-02-07 21:54         ` Eduardo Habkost
  0 siblings, 0 replies; 14+ messages in thread
From: Eduardo Habkost @ 2020-02-07 21:54 UTC (permalink / raw)
  To: Markus Armbruster
  Cc: Peter Maydell, thuth, Cleber Rosa, mdroth, qemu-devel, John Snow

On Sat, Jan 18, 2020 at 07:54:18AM +0100, Markus Armbruster wrote:
> John Snow <jsnow@redhat.com> writes:
> > On 1/17/20 2:07 AM, Markus Armbruster wrote:
> >> John Snow <jsnow@redhat.com> writes:
[...]
> >>> This problem has bitten me *many* times. I'm wondering if there's a
> >>> prescription that isn't just "Wait until we can stipulate 3.6+".
> >> 
> >> No clue.
> >> 
> >> 3.5 EOL is scheduled for 2020-09-13.
> >> https://devguide.python.org/#status-of-python-branches
> >> 
> >> We support 3.5 because we support Debian 9.
> >> 
> >> We'd normally drop support for Debian 9 two years after Debian 10,
> >> i.e. July 2021.  Assuming Debian supports it that far.  Whether they can
> >> truly support Python 3.5 after uptstream EOL seems doubtful.
> >> 
> >
> > We should decide whether we consider Debian LTS to be adequately
> > supported, yes-or-no.
> >
> > We should use a rule of "two years after successor, or End-of-Support,
> > whichever comes first."
> 
> Yes.
> 
> > For Debian, is end of support three years after it comes out, or is it
> > when the LTS is EOL?
> 
> We need to define end-of-support for Debian: is it Debian proper or is
> it Debian LTS?
> 
> <https://wiki.debian.org/DebianOldStable>:
> 
>     Q) How long will security updates be provided?
> 
>     The security team tries to support a stable distribution for about
>     one year after the next stable distribution has been released,
>     except when another stable distribution is released within this
>     year.  It is not possible to support three distributions; supporting
>     two simultaneously is already difficult enough.
> 
> <https://wiki.debian.org/LTS>:
> 
>     Debian Long Term Support (LTS) is a project to extend the lifetime
>     of all Debian stable releases to (at least) 5 years.  Debian LTS is
>     not handled by the Debian security team, but by a separate group of
>     volunteers and companies interested in making it a success.
> 
>     Thus the Debian LTS team takes over security maintenance of the
>     various releases once the Debian Security team stops its work.

As Debian LTS is maintained by a separate group, I interpret
"Debian EOL" as not including LTS.

Supporting Debian 9 in 2020 is already being a burden.
Supporting it until mid-2021 seems pointless.


> 
> Debian 10 "Buster" was released in July 2019.  Debian 9 "Stretch" will
> receive security updates from Debian until mid 2020, i.e. just about
> when Python 3.5 reaches EOL.  LTS will attempt to support it until June
> 2022.
> 
> I think we should give ourselves a bit more flexibility than the
> categorical "Support for the previous major version will be dropped 2
> years after the new major version is released."  At some point, the cost
> of supporting old hosts exceeds the utility.  We should face this
> tradeoff.

Agreed.

-- 
Eduardo



^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, back to index

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-01-16 20:25 [PATCH] qapi: Fix code generation with Python 3.5 Markus Armbruster
2020-01-16 21:17 ` John Snow
2020-01-17  7:07   ` Markus Armbruster
2020-01-17 19:41     ` John Snow
2020-01-18  6:54       ` Markus Armbruster
2020-02-07 21:54         ` Eduardo Habkost
2020-01-18  8:29       ` Philippe Mathieu-Daudé
2020-01-20  5:46         ` [Qemu-devel] Xenial in Travis (was: qapi: Fix code generation with Python 3.5) Thomas Huth
2020-01-17  9:34 ` [PATCH] qapi: Fix code generation with Python 3.5 Thomas Huth
2020-01-17 10:49   ` Markus Armbruster
2020-01-18  8:33     ` Philippe Mathieu-Daudé
2020-01-19 11:23       ` Philippe Mathieu-Daudé
2020-01-20 10:52 ` Alex Bennée
2020-01-20 12:53 ` Peter Maydell

QEMU-Devel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/qemu-devel/0 qemu-devel/git/0.git
	git clone --mirror https://lore.kernel.org/qemu-devel/1 qemu-devel/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 qemu-devel qemu-devel/ https://lore.kernel.org/qemu-devel \
		qemu-devel@nongnu.org
	public-inbox-index qemu-devel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.nongnu.qemu-devel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git